Answers: <p>AWS offers fargate now, which is managed by them rather than being a service like ECS which you ran the servers for and they manage the control plane. Fargate is basically "serverless" ECS. You can use a cloudwatch scheduled even to run something occasionally on a schedule on a fargate ECS cluster. Therefore you're only paying for what you're using. Something to consider is fargate is quite a bit more expensive than ECS running on your own EC2 instance, but if you're running in frequently you'll likely see a pretty marginal savings.</p>
  
  <p>Another option would be to automate something similar. So create an ECS cluster with an ASG backing it, and adjust scale to match ECS demand, and then use cloudwatch to launch the container. ECS can then scale up and down, but this can be tricky because this method is usually used to autoscale ECS clusters based on cloutwatch triggers, so you might have to hack around in cloudwatch and possibly hack together a lambda script, since you're technically not using it the way it's intended. In the end this would probably be the most cost effective, but in your case the cost savings could be negligible.</p>
  
  <p>In my opinion Kubernetes is a ten ton hammer that you'd be using to hang a picture on the wall. It's way overkill.</p>
  
  <p>You can explore those 2 options and see if they are cost effective, and easy enough to manage. Other container orchestration is usually going to be overkill, ECS has the advantage(usually a disadvantage) of being stupid simple and you pay very little and have basically no operational overhead for the control plane.</p>
  
Answers: CC BY-SA 4.0
Answers: 2018-06-25T03:37:51.380Z
Answers: 2018-06-25T03:37:51.380Z
Answers: tsturzl
Answers: <p>The cheapest way is to use <a href="https://devcenter.heroku.com/articles/container-registry-and-runtime" rel="nofollow noreferrer">Heroku's Container Registry</a>.</p>
  
  <p>It's entirely free, easy to use and deploy (You don't even need the heroku CLI , just docker push your image to registry.heroku.com with your auth token)</p>
  
  <p>It may not be the best though but based on your comments I think it should be fine for you, see cons <a href="https://devcenter.heroku.com/articles/container-registry-and-runtime#known-issues-and-limitations" rel="nofollow noreferrer">here</a>.</p>
  
  <p>Another cheap alternative is <a href="https://hyper.sh/hyper/pricing.html" rel="nofollow noreferrer">hyper.sh</a>.</p>
  
Answers: CC BY-SA 4.0
Answers: 2018-06-20T09:06:13.177Z
Answers: 2018-06-20T09:06:13.177Z
Answers: storm
Answers: <p>You can mix the powerful of <a href="https://github.com/Miserlou/Zappa" rel="nofollow noreferrer">Zappa</a> and <a href="https://github.com/timothycrosley/hug" rel="nofollow noreferrer">Hug</a> to convert your code in a serverless <a href="https://docs.aws.amazon.com/lambda/latest/dg/welcome.html" rel="nofollow noreferrer">AWS Lambda</a> function adding only a decorator to your python process main function</p>
  
  <pre><code>import hug
  [...]
  
  @hug.get('/your_endpoint_name')
  def your_function_name():
      """Here goes your code"""
      [...]
      return "Function finished sucesfully"
  </code></pre>
  
  <p>After this you can deploy to AWS Lambda with <code>zappa deploy prod</code> and you will only need to call the returned URL twice a month.</p>
  
Answers: CC BY-SA 4.0
Answers: 2018-06-20T17:40:08.930Z
Answers: 2018-06-20T17:40:08.930Z
Answers: erseco
Answers: <p>Amazon's ECS service is really simple and allows you to schedule container tasks</p>
  
  <p><a href="https://docs.aws.amazon.com/AmazonECS/latest/developerguide/scheduling_tasks.html" rel="nofollow noreferrer">https://docs.aws.amazon.com/AmazonECS/latest/developerguide/scheduling_tasks.html</a></p>
  
Answers: CC BY-SA 4.0
Answers: 2018-06-21T09:35:50.070Z
Answers: 2018-06-21T09:35:50.070Z
Answers: Caledonia91
Answers: <p><a href="https://docs.ansible.com/ansible/latest/modules/ec2_module.html?highlight=ec2" rel="nofollow noreferrer">Ansible ec2</a> module have required functionality (launch instances, runs some tasks and then terminate them) out-of-the-box. And even have sample playbook.</p>
  
Answers: CC BY-SA 4.0
Answers: 2018-06-25T12:44:14.797Z
Answers: 2018-06-25T12:44:14.797Z
Answers: am4
Answers: <p>While I'd normally agree with the other answers that Kubernetes is overkill, <a href="https://kubesail.com" rel="nofollow noreferrer">KubeSail</a> simplifies it as much as possible in order to make it easy to run tasks like yours. There is a free tier that should let you run your job indefinitely. You just need to log in with GitHub, get your <a href="https://kubesail.com/config" rel="nofollow noreferrer">Kube config</a>, and then you can use the following:</p>
  
  <pre><code>apiVersion: batch/v1
  kind: Job
  metadata:
    name: my-python-script
  spec:
    template:
      spec:
        containers:
        - name: my-python-script
          image: asciimo/my-python-image
  </code></pre>
  
  <p>Just save out the above to <code>my-job.yaml</code>, replacing <code>image: asciimo/my-python-image</code> with the name of your image on dockerhub (or another registry), then run</p>
  
  <pre><code>kubectl apply -f my-job.yaml
  </code></pre>
  
  <p>If you wanted to learn more about Jobs, the Kubernetes docs contain a lot more info and options: <a href="https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/" rel="nofollow noreferrer">https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/</a></p>
  
  <p><em>Full disclosure, I am one of the KubeSail Founders</em></p>
  
Answers: CC BY-SA 4.0
Answers: 2019-05-07T18:29:14.157Z
Answers: 2019-05-07T19:01:18.023Z
Answers: 2019-05-07T19:01:18.023Z
Answers: pastudan
Body: <p>I have a dockerized Python process that a client would like to run once or twice a month. It calls a number of Google APIs and stores the results in a Google Sheet. It does not accept network requests. I want to provide one "magic button" that will spin up the Docker container and then shut everything down.</p>
  
  <p>Some background: I had previously deployed this to an AWS EC2 micro instance via Docker Cloud. My client got a Docker Cloud account and was able to run the container on demand, relatively painlessly. There are two problems with this workflow:</p>
  
  <ol>
  <li>The provisioned EC2 instance (Docker host) was running 24/7, whether
  the Docker container was run up or not. This became expensive.</li>
  <li>Docker cloud shut down a month ago.</li>
  </ol>
  
  <p>Alternatives seem to be Google and AWS Kubernetes services. My reservation is that they will be too complicated for my client to use. Advice?</p>
  
Comments: CC BY-SA 4.0
Comments: 2018-06-19T21:19:43.767Z
Comments: How long does this process run for?
Comments: Preston Martin
Comments: CC BY-SA 4.0
Comments: 2018-06-19T21:23:59.350Z
Comments: Bake your python code into an AWS lambda
Comments: Tensibai
Comments: CC BY-SA 4.0
Comments: 2018-06-19T23:57:50.857Z
Comments: @PrestonM, about 25 minutes of hundreds of throttled network API calls.
Comments: asciimo
Comments: CC BY-SA 4.0
Comments: 2018-06-20T00:00:21.283Z
Comments: @Tensibai, I considered that, but my client would need an AWS account and would need to learn how to launch the lambda. Definitely an option, but I already have this fully functional, portable Docker container that I *should* be able to simply throw up onto the cloud :)
Comments: asciimo
Comments: CC BY-SA 4.0
Comments: 2018-06-20T05:03:50.010Z
Comments: I must misunderstand the question then, I don't really get where (who's ec2 machine) it was running before
Comments: Tensibai
Comments: CC BY-SA 4.0
Comments: 2018-06-28T13:27:24.970Z
Comments: They wouldn’t need to understand Lambda, you could give them a “button” via API Gateway, S3, or any other supported trigger. Having said that, 25 mins is about 5 times too long for a Lambda invocation :P
Comments: Tim Malone
Comments: CC BY-SA 4.0
Comments: 2018-08-09T01:23:24.567Z
Comments: @TimMalone that is an excellent idea, and it's currently my Plan B. Plan A is storm's Heroku Container Registry idea; I'm going to shoot for $0 and no account requirements for my client (they are a non-profit). If that fails or disappoints I'll cough up the couple of dollars a month a Lambda solution would cost, if that.
Comments: asciimo
Comments: CC BY-SA 4.0
Comments: 2018-10-26T21:29:09.207Z
Comments: @TimMalone it turns out that 25 minutes is about 10 minutes too long for a lambda function, which is capped at 900 seconds. I would have to refactor my task to process in parallel, probably over a queue. In the end they went with a completely different, paid solution called Awesome Tables.
Comments: asciimo
Tags: <docker><cloud>
Title: Cheapest and easiest way to run a periodic Docker process
UserDisplayName: asciimo
ThreadText: # Cheapest and easiest way to run a periodic Docker process
  
  (Asked by: username_0 on 2018-06-19)
  
  I have a dockerized Python process that a client would like to run once or twice a month. It calls a number of Google APIs and stores the results in a Google Sheet. It does not accept network requests. I want to provide one "magic button" that will spin up the Docker container and then shut everything down.
  
  Some background: I had previously deployed this to an AWS EC2 micro instance via Docker Cloud. My client got a Docker Cloud account and was able to run the container on demand, relatively painlessly. There are two problems with this workflow:
  
  1. The provisioned EC2 instance (Docker host) was running 24/7, whether
     the Docker container was run up or not. This became expensive.
  2. Docker cloud shut down a month ago.
  
  Alternatives seem to be Google and AWS Kubernetes services. My reservation is that they will be too complicated for my client to use. Advice?
  
  **Tags:** docker, cloud
  
  > **Comments:**
  > **username_1**: How long does this process run for?  
  > **username_2**: Bake your python code into an AWS lambda  
  > **username_0**: @PrestonM, about 25 minutes of hundreds of throttled network API calls.  
  > **username_0**: @username_2, I considered that, but my client would need an AWS account and would need to learn how to launch the lambda. Definitely an option, but I already have this fully functional, portable Docker container that I *should* be able to simply throw up onto the cloud :)  
  > **username_2**: I must misunderstand the question then, I don't really get where (who's ec2 machine) it was running before  
  > **username_3**: They wouldn’t need to understand Lambda, you could give them a “button” via API Gateway, S3, or any other supported trigger. Having said that, 25 mins is about 5 times too long for a Lambda invocation :P  
  > **username_0**: @username_3 that is an excellent idea, and it's currently my Plan B. Plan A is username_5's Heroku Container Registry idea; I'm going to shoot for $0 and no account requirements for my client (they are a non-profit). If that fails or disappoints I'll cough up the couple of dollars a month a Lambda solution would cost, if that.  
  > **username_0**: @username_3 it turns out that 25 minutes is about 10 minutes too long for a lambda function, which is capped at 900 seconds. I would have to refactor my task to process in parallel, probably over a queue. In the end they went with a completely different, paid solution called Awesome Tables.  
  
  ---
  
  ## 6 answers
  
  ### Answer by username_4 on 2018-06-25. Score: 3
  
  AWS offers fargate now, which is managed by them rather than being a service like ECS which you ran the servers for and they manage the control plane. Fargate is basically "serverless" ECS. You can use a cloudwatch scheduled even to run something occasionally on a schedule on a fargate ECS cluster. Therefore you're only paying for what you're using. Something to consider is fargate is quite a bit more expensive than ECS running on your own EC2 instance, but if you're running in frequently you'll likely see a pretty marginal savings.
  
  Another option would be to automate something similar. So create an ECS cluster with an ASG backing it, and adjust scale to match ECS demand, and then use cloudwatch to launch the container. ECS can then scale up and down, but this can be tricky because this method is usually used to autoscale ECS clusters based on cloutwatch triggers, so you might have to hack around in cloudwatch and possibly hack together a lambda script, since you're technically not using it the way it's intended. In the end this would probably be the most cost effective, but in your case the cost savings could be negligible.
  
  In my opinion Kubernetes is a ten ton hammer that you'd be using to hang a picture on the wall. It's way overkill.
  
  You can explore those 2 options and see if they are cost effective, and easy enough to manage. Other container orchestration is usually going to be overkill, ECS has the advantage(usually a disadvantage) of being stupid simple and you pay very little and have basically no operational overhead for the control plane.
  
  
  ---
  
  ### Answer by username_5 on 2018-06-20. Score: 2
  
  The cheapest way is to use [Heroku's Container Registry](https://devcenter.heroku.com/articles/container-registry-and-runtime).
  
  It's entirely free, easy to use and deploy (You don't even need the heroku CLI , just docker push your image to registry.heroku.com with your auth token)
  
  It may not be the best though but based on your comments I think it should be fine for you, see cons [here](https://devcenter.heroku.com/articles/container-registry-and-runtime#known-issues-and-limitations).
  
  Another cheap alternative is [hyper.sh](https://hyper.sh/hyper/pricing.html).
  
  
  ---
  
  ### Answer by username_6 on 2018-06-20. Score: 2
  
  You can mix the powerful of [Zappa](https://github.com/Miserlou/Zappa) and [Hug](https://github.com/timothycrosley/hug) to convert your code in a serverless [AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/welcome.html) function adding only a decorator to your python process main function
  
  ```
  import hug
  [...]
  
  @hug.get('/your_endpoint_name')
  def your_function_name():
      """Here goes your code"""
      [...]
      return "Function finished sucesfully"
  
  ```
  
  After this you can deploy to AWS Lambda with `zappa deploy prod` and you will only need to call the returned URL twice a month.
  
  
  ---
  
  ### Answer by username_9 on 2019-05-07. Score: 1
  
  While I'd normally agree with the other answers that Kubernetes is overkill, [KubeSail](https://kubesail.com) simplifies it as much as possible in order to make it easy to run tasks like yours. There is a free tier that should let you run your job indefinitely. You just need to log in with GitHub, get your [Kube config](https://kubesail.com/config), and then you can use the following:
  
  ```
  apiVersion: batch/v1
  kind: Job
  metadata:
    name: my-python-script
  spec:
    template:
      spec:
        containers:
        - name: my-python-script
          image: username_0/my-python-image
  
  ```
  
  Just save out the above to `my-job.yaml`, replacing `image: username_0/my-python-image` with the name of your image on dockerhub (or another registry), then run
  
  ```
  kubectl apply -f my-job.yaml
  
  ```
  
  If you wanted to learn more about Jobs, the Kubernetes docs contain a lot more info and options: <https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/>
  
  *Full disclosure, I am one of the KubeSail Founders*
  
  
  ---
  
  ### Answer by username_7 on 2018-06-21. Score: 0
  
  Amazon's ECS service is really simple and allows you to schedule container tasks
  
  <https://docs.aws.amazon.com/AmazonECS/latest/developerguide/scheduling_tasks.html>
  
  
  ---
  
  ### Answer by username_8 on 2018-06-25. Score: 0
  
  [Ansible ec2](https://docs.ansible.com/ansible/latest/modules/ec2_module.html?highlight=ec2) module have required functionality (launch instances, runs some tasks and then terminate them) out-of-the-box. And even have sample playbook.
  
  
  ---
  
  