Answers: <p>I had a similar situation in which I wanted to nest other parallel jobs threads inside another parallel one. This code worked for me:</p>
  
  <pre><code>def performDeploymentStages(String node, String app) {
      stage("build") {
          echo "Building the app [${app}] on node [${node}]"
      }
      stage("deploy") {
          echo "Deploying the app ${app}] on node [${node}]"
      }
      stage("test") {
          echo "Testing the app [${app}] on node [${node}]"
      }
  }
  
  pipeline {
      agent {
          label 'master'
      }
      parameters {
          string(name: 'NODES', defaultValue: '1,2,3', description: 'Nodes to build, deploy and test')
          choice(name: 'ENV', choices: 'qa', description: 'Environment')
          string(name: 'APPS', defaultValue: 'app01,app02', description: 'App names')
      }
  
      stages {
          stage('parallel stage') {
              steps {
                  script {
                      def nodes = [:]
                      for (node in params.NODES.tokenize(',')) {
                          def apps = [:]
                          for (app in params.APPS.tokenize(',')) {
                              performDeploymentStages(node, app)
                          }
                          parallel apps
                      }
                      parallel nodes
                  }
              }
          }
      }
  }
  </code></pre>
  
  <p>To fully benefit from parallel run remember to assign enough executors.</p>
  
Answers: CC BY-SA 4.0
Answers: 2019-02-08T05:17:52.907Z
Answers: Does this snippet work without assigning values to the `nodes` and `apps` variables?
Answers: jellenberger
Answers: CC BY-SA 4.0
Answers: 2019-02-15T12:26:37.107Z
Answers: @jellenberger it will not work if these vars will be empty (null/empty string). But it's fine to pass single items (single app, or node) without commas.
Answers: biniosuaf
Answers: CC BY-SA 4.0
Answers: 2024-10-21T14:53:48.430Z
Answers: I think the point is, you're not using parallelization. `parallel apps` and `parallel nodes` can be replaced with `parallel [:]`, or commented out, or removed altogether, without any impact on your pipeline that runs 9 stages sequentially.
Answers: MaratC
Answers: CC BY-SA 4.0
Answers: 2018-06-27T16:35:42.250Z
Answers: 2018-06-27T16:35:42.250Z
Answers: biniosuaf
Answers: <p>Working off of the existing answers and the Jenkins <a href="https://www.jenkins.io/doc/pipeline/examples/#parallel-from-grep" rel="nofollow noreferrer">example documentation</a> I came up with the following nested parallel solution. The <a href="https://plugins.jenkins.io/pipeline-stage-view" rel="nofollow noreferrer">Stage View (2.19)</a> plugin and <a href="https://plugins.jenkins.io/blueocean" rel="nofollow noreferrer">Blue Ocean (1.24.5)</a> do not display them as you would hope, so I used the <a href="https://plugins.jenkins.io/groovy-postbuild/" rel="nofollow noreferrer">Groovy Postbuild (2.5)</a> plugin (whitelisted) to show a result summary on the build page of the Classic View (the Badge plugin may work as well).</p>
  <p>Current environment is using Jenkins LTS 2.277.2, a main &quot;trigger&quot; job, and 4 downstream jobs that build the pieces of my application, all of which are Multibranch pipelines.</p>
  <p>I added a method to trigger my downstream Multibranch Pipeline jobs, since they all do the same thing. This is at the end of my Jenkinsfile.</p>
  <pre class="lang-java prettyprint-override"><code>// In case you want to work on the results of the downstream jobs
  buildResults = []
  
  // Common code to trigger a downstream job
  // Outputs a URL that you can follow in Blue Ocean and Classic View
  def performTriggerJob(String jobID) {
      // We need to wrap what we return in a Groovy closure, or else it's invoked
      // when this method is called, not when we pass it to parallel.
      // To do this, you need to wrap the code below in { }, and either return
      // that explicitly, or use { -&gt; } syntax.
      return {
          GIT_BRANCH_FIXUP = env.BRANCH_NAME.replace(&quot;/&quot;,&quot;%2F&quot;)
          // propagate: false so we can work after it returns. Try/catch will also work
          res = build( job: &quot;../${jobID}/${GIT_BRANCH_FIXUP}&quot;, propagate: false)
          echo &quot;${jobID}: ${res.getResult()} - ${res.buildVariables.get('RUN_DISPLAY_URL')}&quot;
          // Add to buildResults
          buildResults &lt;&lt; res
  
          text = &quot;Downstream Run Result: &lt;a href=\&quot;${res.buildVariables.get('RUN_DISPLAY_URL')}\&quot;&gt;${jobID} - ${res.getResult()}&lt;/a&gt;&quot;
  
          if (res.getResult().equals(&quot;SUCCESS&quot;)) {
              manager.createSummary(&quot;green.gif&quot;).appendText(text)
          } else {
              manager.createSummary(&quot;red.png&quot;).appendText(text)
              error 'JOB FAILED' // this fails the stage
          }
      }
  }
  </code></pre>
  <p>Then in the main pipeline of the main Triggering job, I use <code>script</code> blocks to start the nested parallel jobs.</p>
  <pre class="lang-java prettyprint-override"><code>pipeline {
  agent any
  stages {
  stage('Build Nested parallel') {
     steps {
        script {
           // Runs all these builds in parallel, the 2 nested Jobs
           // have a requirement on the cmake build, but can also run in
           // parallel after that finishes. Declarative cannot nest, but
           // Scripted can.
  
           // This is a map of 'name': action
           // performTriggerJob returns a Closure which is why this is just a method &quot;call&quot;
           // as Groovy will invoke .call() on parallel 'actions' automatically
           def builds = [
              &quot;subJob1&quot;: performTriggerJob('subJob1'),
              &quot;subJob2&quot;: performTriggerJob('subJob2'),
              &quot;subJob3&quot;: { script {
                 // Since we have more work to do than just trigger a job,
                 // we have to .call() this Closure directly in the script block to get it to run
                 performTriggerJob('subJob3').call()
  
                 // You can also pass a map directly into the parallel 
                 parallel (
                    'nestedJob1': performTriggerJob('nestedJob1'),
                    'nestedJob2': performTriggerJob('nestedJob2')
                 )
              } }
           ]
           parallel builds
        }
     }
  }
  // Other stages
  }
  }
  </code></pre>
  <p>Since <code>performTriggerJob</code> adds the results of the jobs to the variable <code>buildResults</code> we can do something with all the results once everything is done. I copy all artifacts from the last successful build and then zip them up so they're available on the trigger job.</p>
  <p><code>buildResults</code> is a list of (what I believe are) <a href="https://javadoc.jenkins.io/plugin/workflow-support/org/jenkinsci/plugins/workflow/support/steps/build/RunWrapper.html" rel="nofollow noreferrer">RunWrapper</a> objects, so the methods in this javadoc are what you can call on this object.</p>
  <pre class="lang-java prettyprint-override"><code>// Other stages
  stage(&quot;Post steps&quot;) {
  steps {
  script {
     for( finishedBuildObj in buildResults ) {
        // Obj to use. Could be this build or the last successful build
        buildObj       = finishedBuildObj
        projectName    = buildObj.getFullProjectName()
        if (buildObj.getResult() != &quot;SUCCESS&quot;) {
           buildObj = buildObj.getPreviousSuccessfulBuild()
        }
        copyArtifacts(
           projectName: projectName,
           // Allow this build to continue even if no build is found matching the &quot;Which build&quot; condition,
           // the build's workspace does not exist or is inaccessible, or no artifacts are found matching the specified pattern.
           // By default this build step fails the build if no artifacts are copied.
           optional: true,
           // Ignore the directory structure of the artifacts in the source project and copy all matching artifacts directly into the specified target directory.
           // By default the artifacts are copied in the same directory structure as the source project.
           flatten: true,
           // Target base directory for copy, or leave blank to use the workspace.
           // Directory (and parent directories, if any) will be created if needed.
           //May contain references to build parameters like $PARAM.
           target: 'allArtifacts',
           fingerprintArtifacts: true,
           // How to select the build to copy artifacts from, such as latest successful or stable build, or latest &quot;keep forever&quot; build. Other plugins may provide additional selections.
           // The build number of the selected build will be recorded in the environment for later build steps to reference. For details, see the help of &quot;Result variable suffix&quot; in &quot;Advanced&quot; section.
           selector: specific(buildObj.getNumber().toString())
        )
     }
  }}}
  </code></pre>
  
Answers: CC BY-SA 4.0
Answers: 2021-04-21T15:48:07.837Z
Answers: 2021-04-21T15:48:07.837Z
Answers: KymikoLoco
Answers: <p>Based on the comments to my question, and some basic testing the following seems to work:</p>
  
  <ul>
  <li>Use nested parallel statements. (Thanks <a href="https://devops.stackexchange.com/users/362/lawnmowerlatte">@lawnmowerlatte</a>) </li>
  <li>Although today (May 15th 2017) Jenkins Blue ocean does not visualise this properly. <a href="https://issues.jenkins-ci.org/browse/JENKINS-38442" rel="noreferrer">Possible solutions discussed here</a></li>
  </ul>
  
Answers: CC BY-SA 4.0
Answers: 2020-05-25T17:34:07.120Z
Answers: Jenkins pipeline just told me: `"matrix" or "parallel" cannot be nested inside another "matrix" or "parallel"` - So I'm afraid this answer isn't  universally valid.
Answers: Rick Moritz
Answers: CC BY-SA 4.0
Answers: 2020-12-04T17:44:49.097Z
Answers: @RickMoritz As you can see from biniosuaf answer: it’s possible if you move to scripted pipelines. It seems, if you want to remain in the declarative pipeline world, you are stuck without any parallel pipelines.
Answers: Vser
Answers: CC BY-SA 3.0
Answers: 2017-05-15T08:19:04.590Z
Answers: 2017-05-15T08:19:04.590Z
Answers: Bert Goethals
Body: <p>I've been interested in converting our bespoke Jenkins integrations into a pipeline. However, I can't seem to figure out how to do it.</p>
  
  <p>Can anyone help me with the Jenkins script that could do the following?</p>
  
  <pre><code>1---2---3-----------9---10
      |           |
      |---4-------|
      |           |
      |---5---6---|
          |       |
          |---7---|
  
  1: Start pipeline
  10: End pipeline
  5: Build some files
     * needed by 6, 7,
     * needed as artifacts at the end
  2, 3, 4, 6, 7: Have jUnit result files, should be available at end of
     test (somewhere), even if one failed
  </code></pre>
  
  <p>Is this even possible? Or should I just join after 3, 4, 5? Like this:</p>
  
  <pre><code>1---2---3-------6-------9---10
      |       |   |   |
      |---4---|   7---|
      |       |
      |---5---|
  </code></pre>
  
Comments: CC BY-SA 3.0
Comments: 2017-04-21T09:32:34.627Z
Comments: I'd vote for second option, sounds less likely to break and easier to extend if the need arise.
Comments: Tensibai
Comments: CC BY-SA 3.0
Comments: 2017-04-21T13:41:40.197Z
Comments: @RichardSlater I've built the second option. As that is pretty straight forward. 
  However, I'd like to have the first one running as well, as job 3 and 4 take a bit more time than 5. So the total build time in the first case would be shorter.
Comments: Bert Goethals
Comments: CC BY-SA 3.0
Comments: 2017-04-23T14:41:13.980Z
Comments: possibly being dense, but i haven't understood what the more complex version adds yet. can you document what "9" does and what it depends on? there is no 8, in case you want to revise and add that. :)
Comments: burnettk
Comments: CC BY-SA 3.0
Comments: 2017-04-23T16:58:08.430Z
Comments: Why aren't 3,4,5 6 and 7 separate jenkins jobs that are started by the scripts in the previous jobs?
Comments: avi
Comments: CC BY-SA 3.0
Comments: 2017-04-25T08:12:53.947Z
Comments: @KevinBurnett The complex version allows  6 & 7 to start before 3 & 4 are done. Indeed, there is no 8. It should be at the same level of 7.
Comments: Bert Goethals
Comments: CC BY-SA 3.0
Comments: 2017-04-25T08:13:45.160Z
Comments: @avi I'm really trying to use the Pipeline ability here...
Comments: Bert Goethals
Comments: CC BY-SA 3.0
Comments: 2017-04-26T12:21:54.947Z
Comments: @BertGoethals You can use Pipeline and still have different jobs. It's possible I'm missing something here.
Comments: avi
Comments: CC BY-SA 3.0
Comments: 2017-05-04T16:13:07.837Z
Comments: I was trying to do something similar a few days ago. I could get it to work with nested `parallel` commands, but the flow didn't look right in BlueOcean.
Comments: lawnmowerlatte
Comments: CC BY-SA 3.0
Comments: 2017-05-11T12:05:20.143Z
Comments: @BertGoethals Can't you start another pipeline doing 5 and then 6 and 7 from your main pipeline ?
Comments: Tensibai
Tags: <jenkins><jenkins-pipeline><build-pipeline><jenkinsfile>
Title: How to build a complex parallel Jenkins pipeline?
UserDisplayName: Bert Goethals
ThreadText: # How to build a complex parallel Jenkins pipeline?
  
  (Asked by: username_0 on 2017-04-21)
  
  I've been interested in converting our bespoke Jenkins integrations into a pipeline. However, I can't seem to figure out how to do it.
  
  Can anyone help me with the Jenkins script that could do the following?
  
  ```
  1---2---3-----------9---10
      |           |
      |---4-------|
      |           |
      |---5---6---|
          |       |
          |---7---|
  
  1: Start pipeline
  10: End pipeline
  5: Build some files
     * needed by 6, 7,
     * needed as artifacts at the end
  2, 3, 4, 6, 7: Have jUnit result files, should be available at end of
     test (somewhere), even if one failed
  
  ```
  
  Is this even possible? Or should I just join after 3, 4, 5? Like this:
  
  ```
  1---2---3-------6-------9---10
      |       |   |   |
      |---4---|   7---|
      |       |
      |---5---|
  
  ```
  
  **Tags:** jenkins, jenkins-pipeline, build-pipeline, jenkinsfile
  
  > **Comments:**
  > **username_1**: I'd vote for second option, sounds less likely to break and easier to extend if the need arise.  
  > **username_0**: @RichardSlater I've built the second option. As that is pretty straight forward.  However, I'd like to have the first one running as well, as job 3 and 4 take a bit more time than 5. So the total build time in the first case would be shorter.  
  > **username_2**: possibly being dense, but i haven't understood what the more complex version adds yet. can you document what "9" does and what it depends on? there is no 8, in case you want to revise and add that. :)  
  > **username_3**: Why aren't 3,4,5 6 and 7 separate jenkins jobs that are started by the scripts in the previous jobs?  
  > **username_0**: @KevinBurnett The complex version allows  6 & 7 to start before 3 & 4 are done. Indeed, there is no 8. It should be at the same level of 7.  
  > **username_0**: @username_3 I'm really trying to use the Pipeline ability here...  
  > **username_3**: @username_0 You can use Pipeline and still have different jobs. It's possible I'm missing something here.  
  > **username_4**: I was trying to do something similar a few days ago. I could get it to work with nested `parallel` commands, but the flow didn't look right in BlueOcean.  
  > **username_1**: @username_0 Can't you start another pipeline doing 5 and then 6 and 7 from your main pipeline ?  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_0 on 2017-05-15. Score: 11 | Accepted answer
  
  Based on the comments to my question, and some basic testing the following seems to work:
  
  * Use nested parallel statements. (Thanks [@username_4](https://devops.stackexchange.com/users/362/username_4))
  * Although today (May 15th 2017) Jenkins Blue ocean does not visualise this properly. [Possible solutions discussed here](https://issues.jenkins-ci.org/browse/JENKINS-38442)
  
  > **Comments:**
  > **username_9**: Jenkins pipeline just told me: `"matrix" or "parallel" cannot be nested inside another "matrix" or "parallel"` - So I'm afraid this answer isn't  universally valid.  
  > **username_10**: @username_9 As you can see from username_5 answer: it’s possible if you move to scripted pipelines. It seems, if you want to remain in the declarative pipeline world, you are stuck without any parallel pipelines.  
  
  ---
  
  ### Answer by username_5 on 2018-06-27. Score: 5
  
  I had a similar situation in which I wanted to nest other parallel jobs threads inside another parallel one. This code worked for me:
  
  ```
  def performDeploymentStages(String node, String app) {
      stage("build") {
          echo "Building the app [${app}] on node [${node}]"
      }
      stage("deploy") {
          echo "Deploying the app ${app}] on node [${node}]"
      }
      stage("test") {
          echo "Testing the app [${app}] on node [${node}]"
      }
  }
  
  pipeline {
      agent {
          label 'master'
      }
      parameters {
          string(name: 'NODES', defaultValue: '1,2,3', description: 'Nodes to build, deploy and test')
          choice(name: 'ENV', choices: 'qa', description: 'Environment')
          string(name: 'APPS', defaultValue: 'app01,app02', description: 'App names')
      }
  
      stages {
          stage('parallel stage') {
              steps {
                  script {
                      def nodes = [:]
                      for (node in params.NODES.tokenize(',')) {
                          def apps = [:]
                          for (app in params.APPS.tokenize(',')) {
                              performDeploymentStages(node, app)
                          }
                          parallel apps
                      }
                      parallel nodes
                  }
              }
          }
      }
  }
  
  ```
  
  To fully benefit from parallel run remember to assign enough executors.
  
  > **Comments:**
  > **username_6**: Does this snippet work without assigning values to the `nodes` and `apps` variables?  
  > **username_5**: @username_6 it will not work if these vars will be empty (null/empty string). But it's fine to pass single items (single app, or node) without commas.  
  > **username_7**: I think the point is, you're not using parallelization. `parallel apps` and `parallel nodes` can be replaced with `parallel [:]`, or commented out, or removed altogether, without any impact on your pipeline that runs 9 stages sequentially.  
  
  ---
  
  ### Answer by username_8 on 2021-04-21. Score: 2
  
  Working off of the existing answers and the Jenkins [example documentation](https://www.jenkins.io/doc/pipeline/examples/#parallel-from-grep) I came up with the following nested parallel solution. The [Stage View (2.19)](https://plugins.jenkins.io/pipeline-stage-view) plugin and [Blue Ocean (1.24.5)](https://plugins.jenkins.io/blueocean) do not display them as you would hope, so I used the [Groovy Postbuild (2.5)](https://plugins.jenkins.io/groovy-postbuild/) plugin (whitelisted) to show a result summary on the build page of the Classic View (the Badge plugin may work as well).
  
  Current environment is using Jenkins LTS 2.277.2, a main "trigger" job, and 4 downstream jobs that build the pieces of my application, all of which are Multibranch pipelines.
  
  I added a method to trigger my downstream Multibranch Pipeline jobs, since they all do the same thing. This is at the end of my Jenkinsfile.
  
  ```java
  // In case you want to work on the results of the downstream jobs
  buildResults = []
  
  // Common code to trigger a downstream job
  // Outputs a URL that you can follow in Blue Ocean and Classic View
  def performTriggerJob(String jobID) {
      // We need to wrap what we return in a Groovy closure, or else it's invoked
      // when this method is called, not when we pass it to parallel.
      // To do this, you need to wrap the code below in { }, and either return
      // that explicitly, or use { -> } syntax.
      return {
          GIT_BRANCH_FIXUP = env.BRANCH_NAME.replace("/","%2F")
          // propagate: false so we can work after it returns. Try/catch will also work
          res = build( job: "../${jobID}/${GIT_BRANCH_FIXUP}", propagate: false)
          echo "${jobID}: ${res.getResult()} - ${res.buildVariables.get('RUN_DISPLAY_URL')}"
          // Add to buildResults
          buildResults << res
  
          text = "Downstream Run Result: <a href=\"${res.buildVariables.get('RUN_DISPLAY_URL')}\">${jobID} - ${res.getResult()}</a>"
  
          if (res.getResult().equals("SUCCESS")) {
              manager.createSummary("green.gif").appendText(text)
          } else {
              manager.createSummary("red.png").appendText(text)
              error 'JOB FAILED' // this fails the stage
          }
      }
  }
  
  ```
  
  Then in the main pipeline of the main Triggering job, I use `script` blocks to start the nested parallel jobs.
  
  ```java
  pipeline {
  agent any
  stages {
  stage('Build Nested parallel') {
     steps {
        script {
           // Runs all these builds in parallel, the 2 nested Jobs
           // have a requirement on the cmake build, but can also run in
           // parallel after that finishes. Declarative cannot nest, but
           // Scripted can.
  
           // This is a map of 'name': action
           // performTriggerJob returns a Closure which is why this is just a method "call"
           // as Groovy will invoke .call() on parallel 'actions' automatically
           def builds = [
              "subJob1": performTriggerJob('subJob1'),
              "subJob2": performTriggerJob('subJob2'),
              "subJob3": { script {
                 // Since we have more work to do than just trigger a job,
                 // we have to .call() this Closure directly in the script block to get it to run
                 performTriggerJob('subJob3').call()
  
                 // You can also pass a map directly into the parallel 
                 parallel (
                    'nestedJob1': performTriggerJob('nestedJob1'),
                    'nestedJob2': performTriggerJob('nestedJob2')
                 )
              } }
           ]
           parallel builds
        }
     }
  }
  // Other stages
  }
  }
  
  ```
  
  Since `performTriggerJob` adds the results of the jobs to the variable `buildResults` we can do something with all the results once everything is done. I copy all artifacts from the last successful build and then zip them up so they're available on the trigger job.
  
  `buildResults` is a list of (what I believe are) [RunWrapper](https://javadoc.jenkins.io/plugin/workflow-support/org/jenkinsci/plugins/workflow/support/steps/build/RunWrapper.html) objects, so the methods in this javadoc are what you can call on this object.
  
  ```java
  // Other stages
  stage("Post steps") {
  steps {
  script {
     for( finishedBuildObj in buildResults ) {
        // Obj to use. Could be this build or the last successful build
        buildObj       = finishedBuildObj
        projectName    = buildObj.getFullProjectName()
        if (buildObj.getResult() != "SUCCESS") {
           buildObj = buildObj.getPreviousSuccessfulBuild()
        }
        copyArtifacts(
           projectName: projectName,
           // Allow this build to continue even if no build is found matching the "Which build" condition,
           // the build's workspace does not exist or is inaccessible, or no artifacts are found matching the specified pattern.
           // By default this build step fails the build if no artifacts are copied.
           optional: true,
           // Ignore the directory structure of the artifacts in the source project and copy all matching artifacts directly into the specified target directory.
           // By default the artifacts are copied in the same directory structure as the source project.
           flatten: true,
           // Target base directory for copy, or leave blank to use the workspace.
           // Directory (and parent directories, if any) will be created if needed.
           //May contain references to build parameters like $PARAM.
           target: 'allArtifacts',
           fingerprintArtifacts: true,
           // How to select the build to copy artifacts from, such as latest successful or stable build, or latest "keep forever" build. Other plugins may provide additional selections.
           // The build number of the selected build will be recorded in the environment for later build steps to reference. For details, see the help of "Result variable suffix" in "Advanced" section.
           selector: specific(buildObj.getNumber().toString())
        )
     }
  }}}
  
  ```
  
  
  ---
  
  