Answers: <p>AWS MWAA uses Apache Airflow to create workflows and DAGS (Directed Acyclic Graphs) with Python to orchestrate complex, dependent tasks. AWS Batch is, as the name states, a batch processing service that utilizes docker containers. Batch allows you to manage instance types and container sizes to fine tune costs of workloads. Batch unlike MWAA does not utilize any orchestration service, so apart from the queue that it utilizes, there is no way to control the workflow of tasks.</p>
  <p>If you have a job that requires <a href="https://docs.aws.amazon.com/mwaa/latest/userguide/what-is-mwaa.html#workflows" rel="nofollow noreferrer">complex workflows</a>, AWS MWAA will reduce the complexity of managing those workflows. If you require large scale containerized processing for image analysis, data conversion, etc., AWS Batch will allow you to easily control your workloads to optimize speed and cost.</p>
  <p>It should also be noted that <a href="https://aws.amazon.com/step-functions/?step-functions.sort-by=item.additionalFields.postDateTime&amp;step-functions.sort-order=desc" rel="nofollow noreferrer">AWS Step Functions</a> is also a very efficient way to orchestrate various types of processing and application tasks. For simple tasks, there is very little overhead and management compared to AWS Batch or MWAA.</p>
  
Answers: CC BY-SA 4.0
Answers: 2022-06-01T17:37:07.343Z
Answers: MWAA uses an Airflow worker fleet managed by Fargate. So it scales automatically, just like AWS Batch. I'm not sure it actually scales to zero. AWS Batch scales to zero, AFAIK. Now, _that's_ a difference. (if OP is cost conscious then this aspect should be looked into)
Answers: peterh
Answers: CC BY-SA 4.0
Answers: 2022-06-01T18:19:20.427Z
Answers: Yep, I agree, that's a great point
Answers: Preston Martin
Answers: CC BY-SA 4.0
Answers: 2022-09-02T15:14:51.797Z
Answers: MWAA does not scale to zero. You pay for idle for each environment.
Answers: Mark Richman
Answers: CC BY-SA 4.0
Answers: 2021-05-24T15:44:19.287Z
Answers: 2021-08-30T14:58:42.297Z
Answers: 2021-08-30T14:58:42.297Z
Answers: Preston Martin
Body: <p>I am trying to get a <code>compare and contrast</code> of various job processing services in AWS like AWS Batch, MWAA (Managed Workflow for Apache Airflow).</p>
  <p>Both are used for processing jobs. Just trying to understand when we should go for Batch and when we should go for MWAA.</p>
  
Tags: <amazon-web-services><apache-airflow>
Title: When to use Apache Airflow (MWAA) in AWS instead of AWS Batch?
UserDisplayName: Biju
ThreadText: # When to use Apache Airflow (MWAA) in AWS instead of AWS Batch?
  
  (Asked by: username_0 on 2021-05-24)
  
  I am trying to get a `compare and contrast` of various job processing services in AWS like AWS Batch, MWAA (Managed Workflow for Apache Airflow).
  
  Both are used for processing jobs. Just trying to understand when we should go for Batch and when we should go for MWAA.
  
  **Tags:** amazon-web-services, apache-airflow
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-05-24. Score: 4
  
  AWS MWAA uses Apache Airflow to create workflows and DAGS (Directed Acyclic Graphs) with Python to orchestrate complex, dependent tasks. AWS Batch is, as the name states, a batch processing service that utilizes docker containers. Batch allows you to manage instance types and container sizes to fine tune costs of workloads. Batch unlike MWAA does not utilize any orchestration service, so apart from the queue that it utilizes, there is no way to control the workflow of tasks.
  
  If you have a job that requires [complex workflows](https://docs.aws.amazon.com/mwaa/latest/userguide/what-is-mwaa.html#workflows), AWS MWAA will reduce the complexity of managing those workflows. If you require large scale containerized processing for image analysis, data conversion, etc., AWS Batch will allow you to easily control your workloads to optimize speed and cost.
  
  It should also be noted that [AWS Step Functions](https://aws.amazon.com/step-functions/?step-functions.sort-by=item.additionalFields.postDateTime&step-functions.sort-order=desc) is also a very efficient way to orchestrate various types of processing and application tasks. For simple tasks, there is very little overhead and management compared to AWS Batch or MWAA.
  
  > **Comments:**
  > **username_2**: MWAA uses an Airflow worker fleet managed by Fargate. So it scales automatically, just like AWS Batch. I'm not sure it actually scales to zero. AWS Batch scales to zero, AFAIK. Now, _that's_ a difference. (if OP is cost conscious then this aspect should be looked into)  
  > **username_1**: Yep, I agree, that's a great point  
  > **username_3**: MWAA does not scale to zero. You pay for idle for each environment.  
  
  ---
  
  