Compare commits
7 Commits
Author | SHA1 | Date | |
---|---|---|---|
ebbba235a1 | |||
949119a0dd | |||
e7cde484d5 | |||
924b8a83f5 | |||
06dfc9c901 | |||
0b860534e2 | |||
31c31544ba |
23
README.md
23
README.md
@@ -25,8 +25,8 @@ k8s = kubernetes
|
||||
byok8s is a command line utility that launches
|
||||
a Snakemake workflow on an existing Kubernetes
|
||||
cluster. This allows you to do something
|
||||
like this (also see the [Installation](installing.md)
|
||||
and [Quickstart](quickstart.md) guides in the
|
||||
like this (also see the [Installation](docs/installing.md)
|
||||
and [Quickstart](docs/quickstart.md) guides in the
|
||||
documentation):
|
||||
|
||||
```
|
||||
@@ -45,12 +45,8 @@ minikube stop
|
||||
```
|
||||
|
||||
## Getting Up and Running
|
||||
Snakemake workflows are provided via a Snakefile by
|
||||
the user. Snakemake runs tasks on the Kubernetes (k8s)
|
||||
cluster. The approach is for the user to provide
|
||||
their own Kubernetes cluster (byok8s = Bring Your
|
||||
|
||||
See the [Quickstart Guide](quickstart.md) to get up and
|
||||
See the [Quickstart Guide](docs/quickstart.md) to get up and
|
||||
running with byok8s.
|
||||
|
||||
## How does byok8s work?
|
||||
@@ -102,11 +98,11 @@ or Google Cloud.
|
||||
But the compute nodes can also be virtual, which is where
|
||||
[minikube](https://github.com/kubernetes/minikube) comes in. It creates a
|
||||
kubernetes cluster that is entirely local and virtual, which makes testing
|
||||
easy. See the [byok8s Minikube Guide](kubernetes_minikube.md) for details
|
||||
easy. See the [byok8s Minikube Guide](docs/kubernetes_minikube.md) for details
|
||||
about how to use minikube with byok8s.
|
||||
|
||||
The Travis CI tests also utilize minikube to run test workflows. See [byok8s
|
||||
Travis Tests](travis_tests.md) for more information.
|
||||
Travis Tests](docs/travis_tests.md) for more information.
|
||||
|
||||
## Cloud Providers
|
||||
|
||||
@@ -122,9 +118,8 @@ We have guides for the following:
|
||||
|
||||
| Cloud Provider | Kubernetes Service | Guide | State |
|
||||
|-----------------------------|---------------------------------|-------------------------------------------------|------------|
|
||||
| Minikube (on AWS EC2) | Minikube | [byok8s Minikube Guide](kubernetes_minikube.md) | Finished |
|
||||
| Google Cloud Platform (GCP) | Google Container Engine (GKE) | [byok8s GCP GKE Guide](kubernetes_gcp.md) | Finished |
|
||||
| Amazon Web Services (AWS) | Elastic Container Service (EKS) | [byok8s AWS EKS Guide](kubernetes_aws.md) | Unfinished |
|
||||
| Digital Ocean (DO) | DO Kubernetes (DOK) | [byok8s DO DOK Guide](kubernetes_dok.md) | Unfinished |
|
||||
| Minikube (on AWS EC2) | Minikube | [byok8s Minikube Guide](docs/kubernetes_minikube.md) | Finished |
|
||||
| Google Cloud Platform (GCP) | Google Container Engine (GKE) | [byok8s GCP GKE Guide](docs/kubernetes_gcp.md) | Finished |
|
||||
| Amazon Web Services (AWS) | Elastic Container Service (EKS) | [byok8s AWS EKS Guide](docs/kubernetes_aws.md) | Unfinished |
|
||||
| Digital Ocean (DO) | DO Kubernetes (DOK) | [byok8s DO DOK Guide](docs/kubernetes_dok.md) | Unfinished |
|
||||
|
||||
Own Kubernetes).
|
||||
|
@@ -1,26 +1,21 @@
|
||||
# Installing byok8s
|
||||
|
||||
byok8s requires two pieces of prerequisite software:
|
||||
byok8s requires pieces of prerequisite software:
|
||||
|
||||
- python (conda)
|
||||
- virtualenv (optional)
|
||||
|
||||
It also requires an AWS S3 bucket to be specified
|
||||
(the bucket must exist and credentials to access it
|
||||
must be provided via environment variables, see the
|
||||
[Quickstart](quickstart.md)).
|
||||
|
||||
Additionally, if you are planning to run byok8s on
|
||||
a local virtual kubernetes cluster, you must install:
|
||||
|
||||
- minikube
|
||||
|
||||
Otherwise, if you are planning on running byok8s on
|
||||
remote kubernetes clusters provided by cloud providers,
|
||||
you must install:
|
||||
Otherwise, if you are planning on running byok8s on remote
|
||||
kubernetes clusters provided by cloud providers
|
||||
or etc., you must install:
|
||||
|
||||
- kubernetes, ***OR***
|
||||
- a cloud provider command line tool (`gcloud`, `aws`)
|
||||
- a cloud provider command line tool (`gcloud`, `aws`)
|
||||
|
||||
## Installing Python
|
||||
|
||||
|
@@ -40,9 +40,9 @@ utility will let you know if it needs APIs enabled for actions. If it can't
|
||||
enable the API for you, it will give you a direct link to the relevant Google
|
||||
Cloud Console page.
|
||||
|
||||
## Google Kubernetes Engine
|
||||
## Google Kubernetes Engine (GKE)
|
||||
|
||||
GKE uses Google Cloud compute nodes to run a kubernetes cluster
|
||||
GKE uses GCP compute nodes to run a kubernetes cluster
|
||||
on Google Cloud infrastructure. It automatically sets up the
|
||||
cluster for you, and allows you to use `kubectl` and `gcloud` to
|
||||
manage and interact with the remote cluster.
|
||||
@@ -255,6 +255,9 @@ Events:
|
||||
Normal Started 6s kubelet, gke-mycluster-default-pool-b44fa389-vh3x Started container
|
||||
```
|
||||
|
||||
Congratulations! You've successfully run an executable Snakemake workflow
|
||||
on a Google Cloud kubernetes cluster!
|
||||
|
||||
Delete the GKE cluster when you are done:
|
||||
|
||||
```
|
||||
|
@@ -81,13 +81,14 @@ To fix the problem with the DNS settings, we have to patch
|
||||
the CoreDNS image being used by `kube-system`.
|
||||
To do that, use the file
|
||||
[`test/fixcoredns.yml`](https://github.com/charlesreid1/2019-snakemake-byok8s/blob/master/test/fixcoredns.yml)
|
||||
[`test/fixcoredns.yml`](https://github.com/charlesreid1/2019-snakemake-byok8s/blob/master/test/fixcoredns.yml)
|
||||
in this repository with `kubectl apply`:
|
||||
|
||||
```plain
|
||||
# Fix the DNS container
|
||||
kubectl apply -f fixcoredns.yml
|
||||
|
||||
# Delete all kube-system containers
|
||||
# Delete all kube-system containers
|
||||
kubectl delete --all pods --namespace kube-system
|
||||
```
|
||||
|
||||
@@ -110,6 +111,6 @@ export AWS_SECRET_ACCESS_KEY="XXXXX"
|
||||
byok8s workflow-alpha params-blue --s3-bucket=mah-bukkit
|
||||
```
|
||||
|
||||
Congratulations! You've just run an executable Snakemake workflow
|
||||
on a minikube kubernetes cluster.
|
||||
Congratulations! You've just run an executable Snakemake workflow
|
||||
on a minikube kubernetes cluster.
|
||||
|
||||
|
Reference in New Issue
Block a user