Answers: <p>It is possible to achieve this using <code>async</code> mode. Please find some references for how to do this below.</p>
  
  <p>Refs:</p>
  
  <ul>
  <li><a href="http://toroid.org/ansible-parallel-dispatch" rel="noreferrer">http://toroid.org/ansible-parallel-dispatch</a></li>
  <li><a href="https://blog.crisp.se/2018/01/27/maxwenzin/how-to-run-ansible-tasks-in-parallel" rel="noreferrer">https://blog.crisp.se/2018/01/27/maxwenzin/how-to-run-ansible-tasks-in-parallel</a></li>
  </ul>
  
  <pre><code>---
  
  - name: Run tasks in parallel
    hosts: localhost
    connection: local
    gather_facts: no
    tasks:
      - name: Pretend to create instances
        command: "sleep {{ item }}"  # Instead of calling a long running operation at a cloud provider, we just sleep.
        with_items:
          - 6
          - 8
          - 7
        register: _create_instances
        async: 600  # Maximum runtime in seconds. Adjust as needed.
        poll: 0  # Fire and continue (never poll)
  
      - name: Wait for creation to finish
        async_status:
          jid: "{{ item.ansible_job_id }}"
        register: _jobs
        until: _jobs.finished
        delay: 5  # Check every 5 seconds. Adjust as you like.
        retries: 10  # Retry up to 10 times. Adjust as needed.
        with_items: "{{ _create_instances.results }}"
  </code></pre>
  
Answers: CC BY-SA 4.0
Answers: 2018-09-17T07:46:27.843Z
Answers: While those links may answer the question if they come to break there's nothing left in your answer for future readers, please try to showcase how this would help solving the problem with your own words/exemple and leave the links for more detailed informations only.
Answers: Tensibai
Answers: CC BY-SA 4.0
Answers: 2018-09-17T13:31:22.387Z
Answers: yeah, I can't mark this as the answer until A.) I test it and B.) relevant code is placed in here.  But thank you for pointing me in this direction nevertheless.
Answers: Peter Turner
Answers: CC BY-SA 4.0
Answers: 2018-09-19T11:40:24.783Z
Answers: Sorry, I was it hurry :)
Answers: webknjaz -- Слава Україні
Answers: CC BY-SA 4.0
Answers: 2022-02-21T13:03:07.653Z
Answers: @PeterTurner I believe the code has been added since.
Answers: webknjaz -- Слава Україні
Answers: CC BY-SA 4.0
Answers: 2018-09-16T13:44:45.870Z
Answers: 2019-04-11T15:39:03.580Z
Answers: 2019-04-11T15:39:03.580Z
Answers: webknjaz -- Слава Україні
Answers: <p>To answer your question: No, as of now Ansible can't run loops in parallel.</p>
  <p>I'd use <a href="https://linux.die.net/man/8/newusers" rel="nofollow noreferrer"><code>newusers</code></a> instead, which is made for bulk user creation. Create a file with all users in it, copy it over to the host, and run <code>newusers /path/to/user/list</code> in a <code>command</code> or <code>shell</code> task.</p>
  <p>For example:</p>
  <pre><code>- name: Create multiple users
    shell: newusers /path/to/user/list &amp;&amp; touch /path/to/user/ansible-users-added
    args:
      creates: /path/to/user/ansible-users-added
  
  </code></pre>
  
Answers: CC BY-SA 4.0
Answers: 2018-05-27T21:59:43.700Z
Answers: 2021-04-22T05:17:22.303Z
Answers: 2021-04-22T05:17:22.303Z
Answers: simonz
Answers: <p>As @webKnja mentioned this is possible with <code>async</code> mode. I have recently discovered it myself and learned that you can use it in 3 different ways depending on your needs.</p>
  <ol>
  <li><p><strong>Execute and poll</strong> the results, notice the <code>poll:5</code>, This will poll the results every 5 seconds. You may save some time with this method.</p>
  <pre><code> - name: My long runing task
     some_module_name:
       ip: &quot;{{item.fabric}}&quot;
       username: &quot;{{user}}&quot;
       password: &quot;{{password}}&quot;
       secret: &quot;{{secret}}&quot;
     loop: &quot;{{zoning_list}}&quot;
     register: _alias_vc_0
     async: 60
     poll: 5
  </code></pre>
  </li>
  <li><p><strong>Fire and forget</strong> <code>poll: 0</code>, This is very quick option since Ansible it's just shooting out those tasks. The down side is that we don't know what was the outcome of the task i.e. <code>changed: True/False</code>. Of course it's a downside if you care about the feedback ;).</p>
  <pre><code> name: My long runing task
   some_module_name:
     ip: &quot;{{item.fabric}}&quot;
     username: &quot;{{user}}&quot;
     password: &quot;{{password}}&quot;
     secret: &quot;{{secret}}&quot;
   loop: &quot;{{zoning_list}}&quot;
   register: _alias_vc_0
   async: 60
   poll: 0
  </code></pre>
  </li>
  <li><p><strong>Fire and forget with <code>async_status</code></strong>, the syntax for the task is the same as example <strong>2</strong> whowever it will require additional task <code>async_status</code>. This is my favorite since it's relatively fast (faster then normal looping or the <code>execute and poll</code>) and allows you to capture the feedback although will need to deal with new <code>register</code> for your <code>async_task</code>.</p>
  </li>
  </ol>
  <p><code>retries: 20</code> -- how many attempts before failing.</p>
  <p><code>delay: 2</code> -- how many second to wait between polls.</p>
  <pre><code>    - name: My long runing task
        some_module_name:
          ip: &quot;{{item.fabric}}&quot;
          username: &quot;{{user}}&quot;
          password: &quot;{{password}}&quot;
          secret: &quot;{{secret}}&quot;
        loop: &quot;{{zoning_list}}&quot;
        register: _alias_vc_0
        async: 60
        poll: 0
  
  
      - name: Wait for My long running task to finish
        async_status:
          id: &quot;{{ item.ansible_job_id }}&quot;
          #jid: &quot;{{ item.ansible_job_id }}&quot; # ansible version &gt; 2.8
        register: _jobs_alias_vc_0
        retries: 20
        delay: 2
        until: _jobs_alias_vc_0.finished
        loop: &quot;{{_alias_vc_0.results}}&quot;
  </code></pre>
  <p><strong>A word of caution</strong>, depending on the task yo may not be able to use the <code>async</code> option. I had examples where I was interacting with system which was not able to handle multiple requests for the same resource. I found <code>async</code> option best working if I have to perform the same task across multiple hosts. That's where I was able to &quot;save&quot; the most time.</p>
  <p>Since you posted the link to Ansible documentation in the question I'm not going to do that.</p>
  
Answers: CC BY-SA 4.0
Answers: 2019-05-03T18:43:58.763Z
Answers: @chicks you might want to change the `poll` value to 0 in example 3.  
  
  This is an amazing explanation!! Thnx.
Answers: Debanjan Basu
Answers: CC BY-SA 4.0
Answers: 2019-05-03T19:37:48.140Z
Answers: @DebanjanBasu Anybody can make suggested edits.  I might be the one who approves it in the review queues, but you should get credit for the edit itself.
Answers: chicks
Answers: CC BY-SA 4.0
Answers: 2019-05-08T10:32:01.043Z
Answers: One character edits aren't allowed sadly! :(
Answers: Debanjan Basu
Answers: CC BY-SA 4.0
Answers: 2019-05-31T16:24:17.880Z
Answers: Option 3 works great, thanks! One comment though: as of at least Ansible 2.8, `async_status` requires `jid`, not `id`.
Answers: EdwardTeach
Answers: CC BY-SA 4.0
Answers: 2023-03-09T10:14:03.750Z
Answers: Although correct, this method fails when you enable `jinja2_native` unless Ansible includes a [fix](https://github.com/ansible/ansible/pull/79951). Unfortunately, Ansible, up to and including v2.14.3, does not have it.
Answers: Stephan
Answers: CC BY-SA 4.0
Answers: 2023-05-17T07:54:09.200Z
Answers: "I found async option best working if I have to perform the same task across multiple hosts" -- wouldn't that depend on the task? If a simple task is run against a group of hosts for example, it already runs in parallel, i.e. it will start the task on all hosts and wait for all of them to complete. My understanding is that the greatest benefit of async is when running multiple commands within the same task, that otherwise would block eachother.
Answers: pzkpfw
Answers: CC BY-SA 4.0
Answers: 2018-09-20T14:25:34.043Z
Answers: 2021-10-13T18:03:29.107Z
Answers: 2021-10-13T18:03:29.107Z
Answers: MMT
Body: <p>I'm running Ansible 2.2, but can upgrade if it helps. </p>
  
  <p>I saw <a href="https://github.com/ansible/ansible/issues/12086" rel="noreferrer">this</a> and was pretty excited, but it doesn't seem to be in this (or any) version of Ansible documentation.</p>
  
  <p>The problem I'm trying to solve is I've got 1000 users that I need to manage on a Centos box. </p>
  
  <p>It takes quite a while to run this task serially.  And even more annoying, everything shows up as changed because the "expires" command on the user module always marks the thing as changed. </p>
  
  <p><a href="http://docs.ansible.com/ansible/latest/user_guide/playbooks_async.html" rel="noreferrer">this</a> also looked promising, but it took the same amount of time to run each command in the with_items loop and didn't go any faster (I never bothered to wait long enough to get to the end).</p>
  
  <p>Skipping tasks is fast now (a lot faster than it was in Ansible 2.0), if I can't figure out how to make this work in parallel, I think I'll go back and figure out how to skip pointless tasks and if all else fails, I'll write my own module.  But it seems like I should be able to do all this faster in Ansible.</p>
  
  <hr>
  
  <p>This is what I want to run in parallel,  <code>host_authorizations</code> is a list of usernames and other data.  </p>
  
  <pre><code>  - name: Create/modify OS user accounts
      user: name={{ item.username }} group=sshusers shell=/bin/bash home="/home/selinux-modules/{{ item.username }}" state=present expires={{item.expiredate|default(omit)}}
      with_items: "{{ host_authorizations }}"
      tags: full_maintenance
  </code></pre>
  
Comments: CC BY-SA 3.0
Comments: 2018-04-11T18:31:31.037Z
Comments: Please provide a code snippet. Otherwise it is hard to help.
Comments: 030
Comments: CC BY-SA 3.0
Comments: 2018-04-11T18:45:00.417Z
Comments: @030 there's a snippet, I guess it helps a bit for context.  I'm more interested conceptually if there really is a way to run tasks (in a loop) in parallel on the same host.  I know I could do a ton of individual things with async, but not so much with with_items.
Comments: Peter Turner
Comments: CC BY-SA 3.0
Comments: 2018-04-11T20:53:33.330Z
Comments: So basically if 1000 users have to be created then it should be finished as fast as creating just one user. Interesting, why not using something like LDAP?
Comments: 030
Comments: CC BY-SA 3.0
Comments: 2018-04-11T21:30:07.937Z
Comments: @030 that's what I want. These aren't human users, so I need to add public keys for all the enabled users (and expire the rest). LDAP would have worked for part of it, but not the public key part, still I think that would have worked for the annoying part, so that's a good suggestion.
Comments: Peter Turner
Comments: CC BY-SA 3.0
Comments: 2018-04-12T11:12:55.957Z
Comments: It was just an idea, never implemented. 
  
  `ISSUE TYPE
  Feature Idea`
Comments: Levi
Comments: CC BY-SA 3.0
Comments: 2018-04-12T11:32:22.853Z
Comments: Seriously, you're heading toward a path of pain, I don't think anyone handle more than a dozen of accounts with local account base, as soon as the number of users grow, I assume everyone move to a centralized accounting system, usualy some ldap backend (could be active directory) and then set the expiry time and public key as attributes of this central base then use things like sss_ssh_authorizedkeys to let the ssh server get the authorized keys from this central base.
Comments: Tensibai
Comments: CC BY-SA 3.0
Comments: 2018-04-12T13:36:00.480Z
Comments: @tensibai, that's what ansible is for! These aren't real user accounts, they're for automation and driven by a postgres database that I generate a dynamic inventory off of.
Comments: Peter Turner
Comments: CC BY-SA 3.0
Comments: 2018-04-12T14:33:55.713Z
Comments: I disagree this is what ansible is for (hint being it doesn't do bulk user creation/management). I stand by the point that accounts should not be managed on local account bases at large volume (fact they are not human accounts is irrelevant to the problem anyway)
Comments: Tensibai
Comments: CC BY-SA 4.0
Comments: 2023-05-18T18:23:19.947Z
Comments: I've tried the third option with a list of locales to be created using the `community.general.locale_gen` module. I belive it went a little faster than usual, but not that much. I noticed that there was only one `locale_def` process displayed by `top`. May it be that it would take much more advantage of multiple cores or vCPUs if more processes could be spawned, one for each item in the loop? Maybe that would require more connections to be opened from the Ansible Controller to the host?
Comments: Jaume Sabater
Tags: <ansible>
Title: Is there a way to run with_items loops in parallel in Ansible?
UserDisplayName: Peter Turner
ThreadText: # Is there a way to run with_items loops in parallel in Ansible?
  
  (Asked by: username_0 on 2018-04-11)
  
  I'm running Ansible 2.2, but can upgrade if it helps.
  
  I saw [this](https://github.com/ansible/ansible/issues/12086) and was pretty excited, but it doesn't seem to be in this (or any) version of Ansible documentation.
  
  The problem I'm trying to solve is I've got 1000 users that I need to manage on a Centos box.
  
  It takes quite a while to run this task serially. And even more annoying, everything shows up as changed because the "expires" command on the user module always marks the thing as changed.
  
  [this](http://docs.ansible.com/ansible/latest/user_guide/playbooks_async.html) also looked promising, but it took the same amount of time to run each command in the with\_items loop and didn't go any faster (I never bothered to wait long enough to get to the end).
  
  Skipping tasks is fast now (a lot faster than it was in Ansible 2.0), if I can't figure out how to make this work in parallel, I think I'll go back and figure out how to skip pointless tasks and if all else fails, I'll write my own module. But it seems like I should be able to do all this faster in Ansible.
  
  ---
  
  This is what I want to run in parallel, `host_authorizations` is a list of usernames and other data.
  
  ```
    - name: Create/modify OS user accounts
      user: name={{ item.username }} group=sshusers shell=/bin/bash home="/home/selinux-modules/{{ item.username }}" state=present expires={{item.expiredate|default(omit)}}
      with_items: "{{ host_authorizations }}"
      tags: full_maintenance
  
  ```
  
  **Tags:** ansible
  
  > **Comments:**
  > **username_1**: Please provide a code snippet. Otherwise it is hard to help.  
  > **username_0**: @username_1 there's a snippet, I guess it helps a bit for context.  I'm more interested conceptually if there really is a way to run tasks (in a loop) in parallel on the same host.  I know I could do a ton of individual things with async, but not so much with with_items.  
  > **username_1**: So basically if 1000 users have to be created then it should be finished as fast as creating just one user. Interesting, why not using something like LDAP?  
  > **username_0**: @username_1 that's what I want. These aren't human users, so I need to add public keys for all the enabled users (and expire the rest). LDAP would have worked for part of it, but not the public key part, still I think that would have worked for the annoying part, so that's a good suggestion.  
  > **username_2**: It was just an idea, never implemented.   `ISSUE TYPE Feature Idea`  
  > **username_3**: Seriously, you're heading toward a path of pain, I don't think anyone handle more than a dozen of accounts with local account base, as soon as the number of users grow, I assume everyone move to a centralized accounting system, usualy some ldap backend (could be active directory) and then set the expiry time and public key as attributes of this central base then use things like sss_ssh_authorizedkeys to let the ssh server get the authorized keys from this central base.  
  > **username_0**: @username_3, that's what ansible is for! These aren't real user accounts, they're for automation and driven by a postgres database that I generate a dynamic inventory off of.  
  > **username_3**: I disagree this is what ansible is for (hint being it doesn't do bulk user creation/management). I stand by the point that accounts should not be managed on local account bases at large volume (fact they are not human accounts is irrelevant to the problem anyway)  
  > **username_4**: I've tried the third option with a list of locales to be created using the `community.general.locale_gen` module. I belive it went a little faster than usual, but not that much. I noticed that there was only one `locale_def` process displayed by `top`. May it be that it would take much more advantage of multiple cores or vCPUs if more processes could be spawned, one for each item in the loop? Maybe that would require more connections to be opened from the Ansible Controller to the host?  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_7 on 2018-09-20. Score: 34
  
  As @webKnja mentioned this is possible with `async` mode. I have recently discovered it myself and learned that you can use it in 3 different ways depending on your needs.
  
  1. **Execute and poll** the results, notice the `poll:5`, This will poll the results every 5 seconds. You may save some time with this method.
  
     ```
      - name: My long runing task
        some_module_name:
          ip: "{{item.fabric}}"
          username: "{{user}}"
          password: "{{password}}"
          secret: "{{secret}}"
        loop: "{{zoning_list}}"
        register: _alias_vc_0
        async: 60
        poll: 5
  
     ```
  2. **Fire and forget** `poll: 0`, This is very quick option since Ansible it's just shooting out those tasks. The down side is that we don't know what was the outcome of the task i.e. `changed: True/False`. Of course it's a downside if you care about the feedback ;).
  
     ```
      name: My long runing task
      some_module_name:
        ip: "{{item.fabric}}"
        username: "{{user}}"
        password: "{{password}}"
        secret: "{{secret}}"
      loop: "{{zoning_list}}"
      register: _alias_vc_0
      async: 60
      poll: 0
  
     ```
  3. **Fire and forget with `async_status`**, the syntax for the task is the same as example **2** whowever it will require additional task `async_status`. This is my favorite since it's relatively fast (faster then normal looping or the `execute and poll`) and allows you to capture the feedback although will need to deal with new `register` for your `async_task`.
  
  `retries: 20` -- how many attempts before failing.
  
  `delay: 2` -- how many second to wait between polls.
  
  ```
      - name: My long runing task
        some_module_name:
          ip: "{{item.fabric}}"
          username: "{{user}}"
          password: "{{password}}"
          secret: "{{secret}}"
        loop: "{{zoning_list}}"
        register: _alias_vc_0
        async: 60
        poll: 0
  
      - name: Wait for My long running task to finish
        async_status:
          id: "{{ item.ansible_job_id }}"
          #jid: "{{ item.ansible_job_id }}" # ansible version > 2.8
        register: _jobs_alias_vc_0
        retries: 20
        delay: 2
        until: _jobs_alias_vc_0.finished
        loop: "{{_alias_vc_0.results}}"
  
  ```
  
  **A word of caution**, depending on the task yo may not be able to use the `async` option. I had examples where I was interacting with system which was not able to handle multiple requests for the same resource. I found `async` option best working if I have to perform the same task across multiple hosts. That's where I was able to "save" the most time.
  
  Since you posted the link to Ansible documentation in the question I'm not going to do that.
  
  > **Comments:**
  > **username_8**: @username_9 you might want to change the `poll` value to 0 in example 3.    This is an amazing explanation!! Thnx.  
  > **username_9**: @username_8 Anybody can make suggested edits.  I might be the one who approves it in the review queues, but you should get credit for the edit itself.  
  > **username_8**: One character edits aren't allowed sadly! :(  
  > **username_10**: Option 3 works great, thanks! One comment though: as of at least Ansible 2.8, `async_status` requires `jid`, not `id`.  
  > **username_11**: Although correct, this method fails when you enable `jinja2_native` unless Ansible includes a [fix](https://github.com/ansible/ansible/pull/79951). Unfortunately, Ansible, up to and including v2.14.3, does not have it.  
  > **username_12**: "I found async option best working if I have to perform the same task across multiple hosts" -- wouldn't that depend on the task? If a simple task is run against a group of hosts for example, it already runs in parallel, i.e. it will start the task on all hosts and wait for all of them to complete. My understanding is that the greatest benefit of async is when running multiple commands within the same task, that otherwise would block eachother.  
  
  ---
  
  ### Answer by username_5 on 2018-09-16. Score: 6
  
  It is possible to achieve this using `async` mode. Please find some references for how to do this below.
  
  Refs:
  
  * <http://toroid.org/ansible-parallel-dispatch>
  * <https://blog.crisp.se/2018/01/27/maxwenzin/how-to-run-ansible-tasks-in-parallel>
  
  ```
  ---
  
  - name: Run tasks in parallel
    hosts: localhost
    connection: local
    gather_facts: no
    tasks:
      - name: Pretend to create instances
        command: "sleep {{ item }}"  # Instead of calling a long running operation at a cloud provider, we just sleep.
        with_items:
          - 6
          - 8
          - 7
        register: _create_instances
        async: 600  # Maximum runtime in seconds. Adjust as needed.
        poll: 0  # Fire and continue (never poll)
  
      - name: Wait for creation to finish
        async_status:
          jid: "{{ item.ansible_job_id }}"
        register: _jobs
        until: _jobs.finished
        delay: 5  # Check every 5 seconds. Adjust as you like.
        retries: 10  # Retry up to 10 times. Adjust as needed.
        with_items: "{{ _create_instances.results }}"
  
  ```
  
  > **Comments:**
  > **username_3**: While those links may answer the question if they come to break there's nothing left in your answer for future readers, please try to showcase how this would help solving the problem with your own words/exemple and leave the links for more detailed informations only.  
  > **username_0**: yeah, I can't mark this as the answer until A.) I test it and B.) relevant code is placed in here.  But thank you for pointing me in this direction nevertheless.  
  > **username_5**: Sorry, I was it hurry :)  
  > **username_5**: @username_0 I believe the code has been added since.  
  
  ---
  
  ### Answer by username_6 on 2018-05-27. Score: 3
  
  To answer your question: No, as of now Ansible can't run loops in parallel.
  
  I'd use [`newusers`](https://linux.die.net/man/8/newusers) instead, which is made for bulk user creation. Create a file with all users in it, copy it over to the host, and run `newusers /path/to/user/list` in a `command` or `shell` task.
  
  For example:
  
  ```
  - name: Create multiple users
    shell: newusers /path/to/user/list && touch /path/to/user/ansible-users-added
    args:
      creates: /path/to/user/ansible-users-added
  
  ```
  
  
  ---
  
  