Answers: <p>If your workloads are <strong>exclusively</strong> ephemeral, the answer is <em>probably not worth the infra management effort</em> of deploying K8S.</p>
  
  <p>Kubernetes is best suited for types of workloads that are continuous, with regular incoming traffic and that require zero downtime. That's why it enables you to do things like rolling updates and implementing advanced traffic management patterns like Canary releases.</p>
  
  <p>While you could have your cluster autoscale down to 0 worker nodes you would still incur in the cost of running the control plane node which requires running pods for the <code>apiserver</code>, the <code>controller-manager</code>, the <code>scheduler</code> and the <code>kube-proxy</code> in addition to running the <code>autoscaler</code> pods and log/metric collectors.</p>
  
  <p>Deploying and destroying the clusters on demand is definitely feasible if you manage the cluster with Infrastructure as Code and design your pipelines so that they can setup and teardown clusters with minimal human intervention (waiting for apiserver to come up, waiting for workers to register and come up, etc). An even more elegant way of approaching it would be with using the <a href="https://github.com/kubernetes-sigs/cluster-api" rel="nofollow noreferrer">Cluster API</a> although this would require a management cluster.</p>
  
  <p>Now, is it worth dealing with all the complexity inherent to K8S only to benefit from the downscaling and only to run occasional workloads? I'd say is probably best to manage your instances as regular autoscaling groups.</p>
  
Answers: CC BY-SA 4.0
Answers: 2020-03-11T19:58:28.300Z
Answers: 2020-03-11T19:58:28.300Z
Answers: davidmontoyago
Body: <p>My team is on the process of migrating all our cloud applications from AWS to GCP, and we were considering using GKE for our main program, until we got the <a href="https://www.reddit.com/r/googlecloud/comments/fdgeyb/new_gke_management_fee_of_010_per_hour/" rel="nofollow noreferrer">news that Google will now charge an hourly management fee of $0.10 for each cluster in GKE</a>.</p>
  
  <p>We still haven't started using GKE, and are now considering if it's really the best option for us. Our workload consists of two process-heavy data-analyses pipelines, which we use to run up to 20 analyses simultaneously in different AWS instances, running those once a week, for around 15min to 3h at most (and perhaps will start running it twice a week on the nearby future). On AWS, we currently spawn several instances of EC2 machines for each pipeline, one for each analyses, so we were considering GKE as we could run several analyses on the same cluster, and let GKE do the scaling for us.</p>
  
  <p>I'm also under the impression that this cost is on top of the cost to have one pod running constantly for every nodepool, is that correct? For every nodepool we create inside our cluster, we'd have to pay for a pod to be running in there 24/7 as well, so that's another 'idle cost' to be considered...</p>
  
  <p>We'd only need to run workloads on the cluster for around 15min to 6h each week, so paying for nodes to be up and running 24/7 seems a bit of a waste, but perhaps I'm not seeing the big picture here, as analyses could be run on the same cluster and save us money? Is there an efficient way to shut GKE down while we're not using it - either the pods or, better yet, the whole cluster itself? That'd save us at the very least 690h of running idle every month.</p>
  
  <p>Would killing and deleting all our clusters as soon as the last analyses is done, and recreating it every week, once new analyses are needed, be something feasible? Would there be a cost involved with that? Is that feasible to be done 'automatically'?</p>
  
Comments: CC BY-SA 4.0
Comments: 2020-03-08T00:16:31.550Z
Comments: Maybe it is worth looking at AWS Fargate? Why migrating from AWS to GC?
Comments: Kaymaz
Comments: CC BY-SA 4.0
Comments: 2020-03-09T16:37:12.627Z
Comments: BigQuery ; Google has localized prices and we can pay in our own currency (reais), while AWS only has the option for us to pay in dollars and we end up paying 20% more in government taxes...
Comments: Guilherme Coppini
Tags: <kubernetes><google-cloud-platform><gcp>
Title: Is Google Kubernetes (GKE) worth it for a program that runs once a week?
UserDisplayName: Guilherme Coppini
ThreadText: # Is Google Kubernetes (GKE) worth it for a program that runs once a week?
  
  (Asked by: username_0 on 2020-03-05)
  
  My team is on the process of migrating all our cloud applications from AWS to GCP, and we were considering using GKE for our main program, until we got the [news that Google will now charge an hourly management fee of $0.10 for each cluster in GKE](https://www.reddit.com/r/googlecloud/comments/fdgeyb/new_gke_management_fee_of_010_per_hour/).
  
  We still haven't started using GKE, and are now considering if it's really the best option for us. Our workload consists of two process-heavy data-analyses pipelines, which we use to run up to 20 analyses simultaneously in different AWS instances, running those once a week, for around 15min to 3h at most (and perhaps will start running it twice a week on the nearby future). On AWS, we currently spawn several instances of EC2 machines for each pipeline, one for each analyses, so we were considering GKE as we could run several analyses on the same cluster, and let GKE do the scaling for us.
  
  I'm also under the impression that this cost is on top of the cost to have one pod running constantly for every nodepool, is that correct? For every nodepool we create inside our cluster, we'd have to pay for a pod to be running in there 24/7 as well, so that's another 'idle cost' to be considered...
  
  We'd only need to run workloads on the cluster for around 15min to 6h each week, so paying for nodes to be up and running 24/7 seems a bit of a waste, but perhaps I'm not seeing the big picture here, as analyses could be run on the same cluster and save us money? Is there an efficient way to shut GKE down while we're not using it - either the pods or, better yet, the whole cluster itself? That'd save us at the very least 690h of running idle every month.
  
  Would killing and deleting all our clusters as soon as the last analyses is done, and recreating it every week, once new analyses are needed, be something feasible? Would there be a cost involved with that? Is that feasible to be done 'automatically'?
  
  **Tags:** kubernetes, google-cloud-platform, gcp
  
  > **Comments:**
  > **username_1**: Maybe it is worth looking at AWS Fargate? Why migrating from AWS to GC?  
  > **username_0**: BigQuery ; Google has localized prices and we can pay in our own currency (reais), while AWS only has the option for us to pay in dollars and we end up paying 20% more in government taxes...  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-03-11. Score: 2
  
  If your workloads are **exclusively** ephemeral, the answer is *probably not worth the infra management effort* of deploying K8S.
  
  Kubernetes is best suited for types of workloads that are continuous, with regular incoming traffic and that require zero downtime. That's why it enables you to do things like rolling updates and implementing advanced traffic management patterns like Canary releases.
  
  While you could have your cluster autoscale down to 0 worker nodes you would still incur in the cost of running the control plane node which requires running pods for the `apiserver`, the `controller-manager`, the `scheduler` and the `kube-proxy` in addition to running the `autoscaler` pods and log/metric collectors.
  
  Deploying and destroying the clusters on demand is definitely feasible if you manage the cluster with Infrastructure as Code and design your pipelines so that they can setup and teardown clusters with minimal human intervention (waiting for apiserver to come up, waiting for workers to register and come up, etc). An even more elegant way of approaching it would be with using the [Cluster API](https://github.com/kubernetes-sigs/cluster-api) although this would require a management cluster.
  
  Now, is it worth dealing with all the complexity inherent to K8S only to benefit from the downscaling and only to run occasional workloads? I'd say is probably best to manage your instances as regular autoscaling groups.
  
  
  ---
  
  