Answers: <p>Be aware, that dynamic build steps could cause some problems in some build steps, e.g. when you call an other job:</p>
  
  <pre><code>pipeline {
      stages {
          stage('Test') {
              steps {
                  script {
                      def tests = [:]
                      for (f in findFiles(glob: '**/html/*.html')) {
                          // Create temp variable, otherwise the name will be the last value of the for loop
                          def name = f
                          tests["${name}"] = {
                              build job: "${name}"
                          }
                      }
                      parallel tests
                  }
              }
          }       
      }
  }
  </code></pre>
  
Answers: CC BY-SA 4.0
Answers: 2020-04-08T10:26:09.370Z
Answers: I don't know why but i used stage and echo inside tests, but it will giving stage name and echo as null until I used a local variable like you have used name above. Do you know why that is? Thanks for your answer though!
Answers: Kaushik
Answers: CC BY-SA 4.0
Answers: 2019-08-12T10:06:36.227Z
Answers: 2019-08-12T10:06:36.227Z
Answers: Christian Gripp
Answers: <p>It's much easier to use scripted Pipelines to do this since you can use arbitrary Groovy, but you should still be able to do this with declarative Pipelines using the <a href="https://jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#code-findfiles-code-find-files-in-the-workspace" rel="nofollow noreferrer"><code>findFiles</code></a> step.</p>
  
Answers: CC BY-SA 3.0
Answers: 2018-01-15T22:52:40.987Z
Answers: 2018-01-15T22:52:40.987Z
Answers: jayhendren
Answers: <p>This also works, if you want to stay within the <code>Declarative Pipeline</code> space</p>
  
  <pre><code>// declare our vars outside the pipeline
  def tests = [:]
  def files
  
  pipeline {
      agent any
      stages {
          stage('1') {
              steps {
                  script {
                      // we've declared the variable, now we give it the values
                      files = findFiles(glob: '**/html/*.html')
                      // Loop through them
                      files.each { f -&gt;
                          // add each object from the 'files' loop to the 'tests' array
                          tests[f] = {
                              // we're already in the script{} block, so do our advanced stuff here
                              echo f.toString()
                          }
                      }
                      // Still within the 'Script' block, run the parallel array object
                      parallel tests
                  }
              }
          }       
      }
  }
  </code></pre>
  
Answers: CC BY-SA 4.0
Answers: 2019-01-29T19:57:39.723Z
Answers: If you want to allocate each parallel task to different Jenkins nodes then simply wrap the actions in a `node {}` block, like this:
  
  `tests[f] = {
      node {
          echo f.toString()
      }
  }`
Answers: primetheus
Answers: CC BY-SA 4.0
Answers: 2018-11-15T01:24:34.300Z
Answers: 2018-12-19T21:45:09.707Z
Answers: 2018-12-19T21:45:09.707Z
Answers: primetheus
Answers: <p>Here is a solution where you can setup the maximum number of concurrent tasks running at once with <code>collate(n)</code>.</p>
  <pre><code>pipeline {
    stages {
      stage(&quot;Parallel&quot;) {
        steps {
          script {
            def list = [&quot;alpha&quot;, &quot;beta&quot;, &quot;gamma&quot;, &quot;delta&quot;, &quot;epsilon&quot;, &quot;zeta&quot;, &quot;eta&quot;, &quot;theta&quot;, &quot;iota&quot;, &quot;kappa&quot;]
  
            // Split the list into batches of 4
            def parallelBatches = list.collate(4);
            // [[&quot;alpha&quot;, &quot;beta&quot;, &quot;gamma&quot;, &quot;delta&quot;], [&quot;epsilon&quot;, &quot;zeta&quot;, &quot;eta&quot;, &quot;theta&quot;], [&quot;iota&quot;, &quot;kappa&quot;]]
  
            // This allows to run a maximum of 4 tasks in parallel (at once)
            for (parallelBatch in parallelBatches) {
              def tasks = [:]
  
              for (task in parallelBatch) {
                tasks[&quot;${task}&quot;] = {
                  stage(&quot;${task}&quot;) {
                    echo '${task}'
                  }
                }
              }
              parallel tasks
  
              // Once the batch is over, the outer loop move onto the next batch
            }
          }
        }
      }
    }
  }
  </code></pre>
  
Answers: CC BY-SA 4.0
Answers: 2022-10-07T08:52:04.587Z
Answers: 2022-10-07T08:52:04.587Z
Answers: shellwhale
Answers: <p>Managed to solve it with the following code:</p>
  
  <pre><code>pipeline {
      agent { label "master"}
      stages {
          stage('1') {
              steps {
                  script {
                      def tests = [:]
                      for (f in findFiles(glob: '**/html/*.html')) {
                          tests["${f}"] = {
                              node {
                                  stage("${f}") {
                                      echo '${f}'
                                  }
                              }
                          }
                      }
                      parallel tests
                  }
              }
          }       
      }
  }
  </code></pre>
  
Answers: CC BY-SA 3.0
Answers: 2018-04-10T16:15:40.517Z
Answers: Please also check out the official Pipeline examples - https://jenkins.io/doc/pipeline/examples/#parallel-multiple-nodes
Answers: phedoreanu
Answers: CC BY-SA 3.0
Answers: 2018-04-10T16:24:56.657Z
Answers: @phedoreanu i'm using declarative pipeline...
Answers: thclpr
Answers: CC BY-SA 3.0
Answers: 2018-04-16T09:38:54.890Z
Answers: @phedoreanu I have rejected your edit, editing code should have good reasons, your comment isn't enough for me to allow an edit of this kind on an answer which was a self solution. I think you should have commented to discuss the matter with the answer author before doing this edit.
Answers: Tensibai
Answers: CC BY-SA 3.0
Answers: 2018-04-16T09:40:48.103Z
Answers: @phedoreanu I you think you have a better derivative work, then please write your own answer and explain why it is better (in error handling, templates, etc.) instead.
Answers: Tensibai
Answers: CC BY-SA 4.0
Answers: 2019-04-11T10:13:41.723Z
Answers: Hi, I figured out the same after a few failing attempts. My only problem right now is that if I put two stage{..} sections inside a node for some reasons the workflow stage chart and Blu Ocean get confused. For instance in the workflow stage chart I get NaNy NaNd and in Blue Ocean I get only the first stage.
Answers: Giuseppe Salvatore
Answers: CC BY-SA 4.0
Answers: 2022-10-06T11:49:10.070Z
Answers: How would you limit this for, let's say 10 concurrents builds running at once ?
Answers: shellwhale
Answers: CC BY-SA 3.0
Answers: 2018-01-16T23:45:09.720Z
Answers: 2018-04-16T09:41:07.233Z
Answers: 2018-04-16T09:41:07.233Z
Answers: thclpr
Body: <p>Currently, I'm going to need an implementation that must find all files within a directory and start a parallel task for every file found.</p>
  
  <p>Is it possible to achieve this using declarative pipelines?</p>
  
  <pre><code>pipeline {
      agent any
      stages {
          stage("test") {
              steps {
                  dir ("file_path") {
                      // find all files with complete path
                      parallel (
                          // execute parallel tasks for each file found.
                          // this must be dynamic
                          }
                      }
                  }
              }
          }
      }
  }
  </code></pre>
  
Comments: CC BY-SA 4.0
Comments: 2019-02-27T18:06:54.747Z
Comments: How can I do if i want to run multiple steps sequentaly and not parallel?
Comments: Frank Escobar
Comments: CC BY-SA 4.0
Comments: 2020-01-29T12:00:57.053Z
Comments: Sure, but that way you cannot generate the parallel tasks dynamically, for example, depending on some files on the repository.
Comments: Raúl Salinas-Monteagudo
Tags: <jenkins><jenkinsfile>
Title: How to properly achieve dynamic parallel action with a declarative pipeline?
UserDisplayName: thclpr
ThreadText: # How to properly achieve dynamic parallel action with a declarative pipeline?
  
  (Asked by: username_0 on 2018-01-12)
  
  Currently, I'm going to need an implementation that must find all files within a directory and start a parallel task for every file found.
  
  Is it possible to achieve this using declarative pipelines?
  
  ```
  pipeline {
      agent any
      stages {
          stage("test") {
              steps {
                  dir ("file_path") {
                      // find all files with complete path
                      parallel (
                          // execute parallel tasks for each file found.
                          // this must be dynamic
                          }
                      }
                  }
              }
          }
      }
  }
  
  ```
  
  **Tags:** jenkins, jenkinsfile
  
  > **Comments:**
  > **username_1**: How can I do if i want to run multiple steps sequentaly and not parallel?  
  > **username_2**: Sure, but that way you cannot generate the parallel tasks dynamically, for example, depending on some files on the repository.  
  
  ---
  
  ## 5 answers
  
  ### Answer by username_0 on 2018-01-16. Score: 40 | Accepted answer
  
  Managed to solve it with the following code:
  
  ```
  pipeline {
      agent { label "master"}
      stages {
          stage('1') {
              steps {
                  script {
                      def tests = [:]
                      for (f in findFiles(glob: '**/html/*.html')) {
                          tests["${f}"] = {
                              node {
                                  stage("${f}") {
                                      echo '${f}'
                                  }
                              }
                          }
                      }
                      parallel tests
                  }
              }
          }       
      }
  }
  
  ```
  
  > **Comments:**
  > **username_8**: Please also check out the official Pipeline examples - https://jenkins.io/doc/pipeline/examples/#parallel-multiple-nodes  
  > **username_0**: @username_8 i'm using declarative pipeline...  
  > **username_9**: @username_8 I have rejected your edit, editing code should have good reasons, your comment isn't enough for me to allow an edit of this kind on an answer which was a self solution. I think you should have commented to discuss the matter with the answer author before doing this edit.  
  > **username_9**: @username_8 I you think you have a better derivative work, then please write your own answer and explain why it is better (in error handling, templates, etc.) instead.  
  > **username_10**: Hi, I figured out the same after a few failing attempts. My only problem right now is that if I put two stage{..} sections inside a node for some reasons the workflow stage chart and Blu Ocean get confused. For instance in the workflow stage chart I get NaNy NaNd and in Blue Ocean I get only the first stage.  
  > **username_7**: How would you limit this for, let's say 10 concurrents builds running at once ?  
  
  ---
  
  ### Answer by username_6 on 2018-11-15. Score: 15
  
  This also works, if you want to stay within the `Declarative Pipeline` space
  
  ```
  // declare our vars outside the pipeline
  def tests = [:]
  def files
  
  pipeline {
      agent any
      stages {
          stage('1') {
              steps {
                  script {
                      // we've declared the variable, now we give it the values
                      files = findFiles(glob: '**/html/*.html')
                      // Loop through them
                      files.each { f ->
                          // add each object from the 'files' loop to the 'tests' array
                          tests[f] = {
                              // we're already in the script{} block, so do our advanced stuff here
                              echo f.toString()
                          }
                      }
                      // Still within the 'Script' block, run the parallel array object
                      parallel tests
                  }
              }
          }       
      }
  }
  
  ```
  
  > **Comments:**
  > **username_6**: If you want to allocate each parallel task to different Jenkins nodes then simply wrap the actions in a `node {}` block, like this:  `tests[f] = {     node {         echo f.toString()     } }`  
  
  ---
  
  ### Answer by username_3 on 2019-08-12. Score: 7
  
  Be aware, that dynamic build steps could cause some problems in some build steps, e.g. when you call an other job:
  
  ```
  pipeline {
      stages {
          stage('Test') {
              steps {
                  script {
                      def tests = [:]
                      for (f in findFiles(glob: '**/html/*.html')) {
                          // Create temp variable, otherwise the name will be the last value of the for loop
                          def name = f
                          tests["${name}"] = {
                              build job: "${name}"
                          }
                      }
                      parallel tests
                  }
              }
          }       
      }
  }
  
  ```
  
  > **Comments:**
  > **username_4**: I don't know why but i used stage and echo inside tests, but it will giving stage name and echo as null until I used a local variable like you have used name above. Do you know why that is? Thanks for your answer though!  
  
  ---
  
  ### Answer by username_5 on 2018-01-15. Score: 2
  
  It's much easier to use scripted Pipelines to do this since you can use arbitrary Groovy, but you should still be able to do this with declarative Pipelines using the [`findFiles`](https://jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#code-findfiles-code-find-files-in-the-workspace) step.
  
  
  ---
  
  ### Answer by username_7 on 2022-10-07. Score: 1
  
  Here is a solution where you can setup the maximum number of concurrent tasks running at once with `collate(n)`.
  
  ```
  pipeline {
    stages {
      stage("Parallel") {
        steps {
          script {
            def list = ["alpha", "beta", "gamma", "delta", "epsilon", "zeta", "eta", "theta", "iota", "kappa"]
  
            // Split the list into batches of 4
            def parallelBatches = list.collate(4);
            // [["alpha", "beta", "gamma", "delta"], ["epsilon", "zeta", "eta", "theta"], ["iota", "kappa"]]
  
            // This allows to run a maximum of 4 tasks in parallel (at once)
            for (parallelBatch in parallelBatches) {
              def tasks = [:]
  
              for (task in parallelBatch) {
                tasks["${task}"] = {
                  stage("${task}") {
                    echo '${task}'
                  }
                }
              }
              parallel tasks
  
              // Once the batch is over, the outer loop move onto the next batch
            }
          }
        }
      }
    }
  }
  
  ```
  
  
  ---
  
  