Answers: <p>You can check out AWS Fargate, I've just used it for a similar use case.</p>
  
  <p>Setup:</p>
  
  <p>Jobs are posted to an SQS queue where a scheduler lambda picks them up and launches an AWS Fargate task.</p>
  
  <p>Caveats:</p>
  
  <ul>
  <li>Lambda handles the SQS messages so you can't hand over the messages to Fargate (Lambda will auto-delete the messages from the queue once the lambda returns successfully), instead you'll need to handle retries via a separate lambda that gets notified whenever a task ends.</li>
  <li>Fargate only supports up to 4 vCPUs</li>
  <li>You need to pass any data to the container either via environment variables or as command line arguments</li>
  </ul>
  
  <p>Launch times are not as fast as lambda, but much better than Batch and EC2 - I typically get my containers within 30 seconds of the lambda launching them.</p>
  
  <p>Dead-letter/retry handling is a bit of a nuisance, but it's not that hard to implement as long as your process is well-behaved. You can easily get the process exit status from the API once the task has finished and you can store the number of retries as a tag on the task since you'll be launching a new task for each retry.</p>
  
  <p>The scheduler and retry handler should be fairly trivial to write and if you need to pass larger amounts of data you can store it on S3 and send the location as an argument to your container.</p>
  
  <p>A similar option - if you don't mind paying for the extra, unused capacity is to use ECS instead of Fargate and have unused capacity in the cluster so you can spawn new containers more or less instantly. This removes the limit on vCPU but you'll instead need to manage the cluster size (can be done via Autoscaling) and pay for the unused capacity.</p>
  
Answers: CC BY-SA 4.0
Answers: 2020-05-04T12:53:53.183Z
Answers: 2020-05-04T12:53:53.183Z
Answers: Raniz
Body: <h2>Goal</h2>
  
  <p>So I'm looking for an AWS product or combination of products to accomplish the following:</p>
  
  <ul>
  <li>Ability to do CPU heavy (non-parallel) calculations (min 8 vCPU's per node)</li>
  <li>Minimum timeout limit of 1800 seconds (AWS Lambda has a limit of 900 seconds)</li>
  <li>Ability to automatically scale up and down to 0/1</li>
  <li>Ability to scale up really fast (&lt;30 seconds)</li>
  <li>Event-driven execution model (one task per node, node gets destroyed after the job is finished)</li>
  <li>Assign the desired vCPU's per "type" (3 predefined types) event/task/job</li>
  </ul>
  
  <p>I'll try to be as detailed as possible in my description of what the job is, the two setups I've tried and what I don't like about them.</p>
  
  <hr>
  
  <h2>Definitions</h2>
  
  <ul>
  <li><p>ROE: Route Optimisation Engine</p></li>
  <li><p>VRP: Vehicle Routing Problem</p>
  
  <p>A single JSON containing all the information regarding the stops, vehicles, time windows, start/end addresses) which needs to be optimised. VRP's can be classified into three different complexities [easy, medium, hard] based on:</p>
  
  <ul>
  <li>number of stops</li>
  <li>restrictions per stop (time window, capacity)</li>
  <li>number of vehicles</li>
  <li>restrictions per vehicle (time window, maximum range, capacity, breaks)</li>
  </ul>
  
  <p>Easy classified VRP's will need less CPU resources and can be solved quicker than for example high classified VRP's.</p></li>
  <li><p>Solution: The best solution for the VRP (the most efficient routes for the VRP)</p></li>
  </ul>
  
  <h2>Setup 1 -  AWS Lambda</h2>
  
  <h3>Diagram</h3>
  
  <p><a href="https://i.sstatic.net/Z3W6r.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Z3W6r.png" alt="enter image description here"></a></p>
  
  <h3>AWS SQS</h3>
  
  <p>Standard type AWS SQS queue for optimisation request messages. Every message that enters triggers the ROE (AWS Lambda).</p>
  
  <h3>AWS Lambda</h3>
  
  <p>The Lambda (3008 MB memory) is triggered by the AWS SQS queue for optimisation request messages and processes alles messages when they are added to the queue. The maximum timeout limit of all AWS Lambda's is 15 minutes.</p>
  
  <h3>Problems</h3>
  
  <ul>
  <li>AWS Lambda does not allow us to select more CPU resources and therefore optimisations of medium complexity VRP's take a long time.</li>
  <li>AWS Lambda maximum timeout settings of 15 minutes makes the usage of a Lambda for doing optimising high complexity VRP impossible</li>
  </ul>
  
  <h2>Setup 2 - AWS Batch</h2>
  
  <h3>Diagram</h3>
  
  <p><a href="https://i.sstatic.net/XHIvO.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/XHIvO.png" alt="enter image description here"></a></p>
  
  <h3>Steps</h3>
  
  <ul>
  <li>WA sends optimisation request to API</li>
  <li>API creates a VRP and stores the VRP in S3</li>
  <li>API evaluates the complexity of the VRP (low, medium or high)</li>
  <li>API creates an AWS Batch Job (with the parameter problemId=123456789)</li>
  <li>API allocates the correct number of vCPU's and Memory (predefined) to that job based on the VRP complexity</li>
  <li>API adds the AWS Batch Job to the correct AWS Batch queue (the queue is defined by the VRP complexity)</li>
  <li>API returns a 200 - OK response to the WA</li>
  <li>AWS Batch Job is taken from the AWS Batch Queue for execution</li>
  <li>AWS Batch Job fetches the VRP from S3 based on the problemId it received during step 4</li>
  <li>AWS Batch Job solves the VRP</li>
  <li>AWS Batch Job stores the solution in S3</li>
  <li>WA requests the solution from the API</li>
  <li>API gets the solution from S3 and returns it to WA</li>
  </ul>
  
  <h3>Problems</h3>
  
  <ul>
  <li>Scaling up takes unacceptably long</li>
  <li>Going from 0 to 1 node takes too long (300+ seconds)</li>
  <li>Going from 1 to X nodes takes too long (60/300+ seconds)</li>
  </ul>
  
Tags: <amazon-web-services><amazon-ecs><aws-lambda><java><serverless>
Title: I want to scale up/down as fast as AWS Lambda but also be able to allocate the vCPU (minimum 8) per task, any advice?
UserDisplayName: narzero
ThreadText: # I want to scale up/down as fast as AWS Lambda but also be able to allocate the vCPU (minimum 8) per task, any advice?
  
  (Asked by: username_0 on 2020-04-24)
  
  ## Goal
  
  So I'm looking for an AWS product or combination of products to accomplish the following:
  
  * Ability to do CPU heavy (non-parallel) calculations (min 8 vCPU's per node)
  * Minimum timeout limit of 1800 seconds (AWS Lambda has a limit of 900 seconds)
  * Ability to automatically scale up and down to 0/1
  * Ability to scale up really fast (<30 seconds)
  * Event-driven execution model (one task per node, node gets destroyed after the job is finished)
  * Assign the desired vCPU's per "type" (3 predefined types) event/task/job
  
  I'll try to be as detailed as possible in my description of what the job is, the two setups I've tried and what I don't like about them.
  
  ---
  
  ## Definitions
  
  * ROE: Route Optimisation Engine
  * VRP: Vehicle Routing Problem
  
    A single JSON containing all the information regarding the stops, vehicles, time windows, start/end addresses) which needs to be optimised. VRP's can be classified into three different complexities [easy, medium, hard] based on:
  
    + number of stops
    + restrictions per stop (time window, capacity)
    + number of vehicles
    + restrictions per vehicle (time window, maximum range, capacity, breaks)
  
    Easy classified VRP's will need less CPU resources and can be solved quicker than for example high classified VRP's.
  * Solution: The best solution for the VRP (the most efficient routes for the VRP)
  
  ## Setup 1 - AWS Lambda
  
  ### Diagram
  
  [![enter image description here](https://i.sstatic.net/Z3W6r.png)](https://i.sstatic.net/Z3W6r.png)
  
  ### AWS SQS
  
  Standard type AWS SQS queue for optimisation request messages. Every message that enters triggers the ROE (AWS Lambda).
  
  ### AWS Lambda
  
  The Lambda (3008 MB memory) is triggered by the AWS SQS queue for optimisation request messages and processes alles messages when they are added to the queue. The maximum timeout limit of all AWS Lambda's is 15 minutes.
  
  ### Problems
  
  * AWS Lambda does not allow us to select more CPU resources and therefore optimisations of medium complexity VRP's take a long time.
  * AWS Lambda maximum timeout settings of 15 minutes makes the usage of a Lambda for doing optimising high complexity VRP impossible
  
  ## Setup 2 - AWS Batch
  
  ### Diagram
  
  [![enter image description here](https://i.sstatic.net/XHIvO.png)](https://i.sstatic.net/XHIvO.png)
  
  ### Steps
  
  * WA sends optimisation request to API
  * API creates a VRP and stores the VRP in S3
  * API evaluates the complexity of the VRP (low, medium or high)
  * API creates an AWS Batch Job (with the parameter problemId=123456789)
  * API allocates the correct number of vCPU's and Memory (predefined) to that job based on the VRP complexity
  * API adds the AWS Batch Job to the correct AWS Batch queue (the queue is defined by the VRP complexity)
  * API returns a 200 - OK response to the WA
  * AWS Batch Job is taken from the AWS Batch Queue for execution
  * AWS Batch Job fetches the VRP from S3 based on the problemId it received during step 4
  * AWS Batch Job solves the VRP
  * AWS Batch Job stores the solution in S3
  * WA requests the solution from the API
  * API gets the solution from S3 and returns it to WA
  
  ### Problems
  
  * Scaling up takes unacceptably long
  * Going from 0 to 1 node takes too long (300+ seconds)
  * Going from 1 to X nodes takes too long (60/300+ seconds)
  
  **Tags:** amazon-web-services, amazon-ecs, aws-lambda, java, serverless
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-05-04. Score: 1
  
  You can check out AWS Fargate, I've just used it for a similar use case.
  
  Setup:
  
  Jobs are posted to an SQS queue where a scheduler lambda picks them up and launches an AWS Fargate task.
  
  Caveats:
  
  * Lambda handles the SQS messages so you can't hand over the messages to Fargate (Lambda will auto-delete the messages from the queue once the lambda returns successfully), instead you'll need to handle retries via a separate lambda that gets notified whenever a task ends.
  * Fargate only supports up to 4 vCPUs
  * You need to pass any data to the container either via environment variables or as command line arguments
  
  Launch times are not as fast as lambda, but much better than Batch and EC2 - I typically get my containers within 30 seconds of the lambda launching them.
  
  Dead-letter/retry handling is a bit of a nuisance, but it's not that hard to implement as long as your process is well-behaved. You can easily get the process exit status from the API once the task has finished and you can store the number of retries as a tag on the task since you'll be launching a new task for each retry.
  
  The scheduler and retry handler should be fairly trivial to write and if you need to pass larger amounts of data you can store it on S3 and send the location as an argument to your container.
  
  A similar option - if you don't mind paying for the extra, unused capacity is to use ECS instead of Fargate and have unused capacity in the cluster so you can spawn new containers more or less instantly. This removes the limit on vCPU but you'll instead need to manage the cluster size (can be done via Autoscaling) and pay for the unused capacity.
  
  
  ---
  
  