Answers: <p><a href="https://en.wikipedia.org/wiki/Single_responsibility_principle" rel="nofollow noreferrer">Single responsibility principle (SRP)</a></p>
  
  <blockquote>
    <p>Martin defines a responsibility as a reason to change, and concludes
    that a class or module should have one, and only one, reason to be
    changed (i.e. rewritten). As an example, consider a module that
    compiles and prints a report. Imagine such a module can be changed for
    two reasons. First, the content of the report could change. Second,
    the format of the report could change. These two things change for
    very different causes; one substantive, and one cosmetic. The single
    responsibility principle says that these two aspects of the problem
    are really two separate responsibilities, and should therefore be in
    separate classes or modules. It would be a bad design to couple two
    things that change for different reasons at different times.</p>
    
    <p>The reason it is important to keep a class focused on a single concern
    is that it makes the class more robust. Continuing with the foregoing
    example, if there is a change to the report compilation process, there
    is greater danger that the printing code will break if it is part of
    the same class.</p>
  </blockquote>
  
  <p><a href="https://softwareengineering.stackexchange.com/a/133406/218283">5-15 lines</a></p>
  
  <blockquote>
    <p>Functions should normally be short, between 5-15 lines is my personal
    "rule of thumb" when coding in Java or C#. This is a good size for
    several reasons:</p>
    
    <ul>
    <li>It fits easily on your screen without scrolling</li>
    <li>It's about the conceptual size that you can hold in your head</li>
    <li>It's meaningful enough to require a function in its own right (as a standalone, meaningful chunk of logic)</li>
    <li>A function smaller than 5 lines is a hint that you are perhaps breaking the code up too much (which makes it harder to read /
    understand if you need to navigate between functions). Either that or
    your're forgetting your special cases / error handling!</li>
    </ul>
  </blockquote>
  
  <p><strong>Personal view</strong></p>
  
  <p>I have the SRP in mind when creating Jenkins stages. This implies that I create a separate stage for checkout, build and publish for example. If a stage becomes too large, i.e. longer than 15 lines, then I create a script and run the script in this stage. One of the benefits is readability, possible to apply unit testing on the script and CI agnostic, i.e. migration to another CI would be possible as well.</p>
  
  <p><a href="https://jenkins.io/doc/book/pipeline/jenkinsfile/" rel="nofollow noreferrer">Example</a></p>
  
  <blockquote>
  <pre><code>pipeline {
      agent any
  
      stages {
          stage('Build') {
              steps {
                  echo 'Building..'
              }
          }
          stage('Test') {
              steps {
                  echo 'Testing..'
              }
          }
          stage('Deploy') {
              steps {
                  echo 'Deploying....'
              }
          }
      }
  }
  </code></pre>
  </blockquote>
  
Answers: CC BY-SA 4.0
Answers: 2019-06-06T19:42:23.647Z
Answers: 2019-06-06T19:48:38.930Z
Answers: 2019-06-06T19:48:38.930Z
Answers: 030
Answers: <p>In addition to being a pipeline visualization tool, stages also identify portions of the pipeline that could be executed more or less independently from each other, potentially on different workers. Do not underestimate the importance of choosing the right pipeline structure.</p>
  
  <p>Stages can be re-executed if/when needed, for example in case of failures caused by environment issues, without having to re-run the already completed upstream stages.</p>
  
  <p>Stages provide ability for pipeline branching. For example a single build stage can produce artifacts that could be used by multiple downstream test stages than can be executed in parallel, for shorter overall pipeline execution time.</p>
  
  <p>Each stage operation can be supported by pools of workers of different types and sizes.</p>
  
  <p>Stages can be conditionally executed. For example an expensive integration test stage (lengthy and/or with a smaller resource pool than the build stage)  could be executed only once in a while rather than for every commit, or only on a release branch and not on the primary development branch.</p>
  
  <p>Since generally the intent is to have the pipeline triggered by each and every commit in the respective branch the size of the pipeline - i.e. the number of stages, their duration and the structure of the pipeline graph (it doesn't necessarily have to be linear) - has a potentially complex relationship with:</p>
  
  <ul>
  <li>the rate and profile of the commits triggering the pipelines</li>
  <li>the pool(s) of resources available to support the operation of the pipeline stage(s)</li>
  <li>the role of the branch, the associated balance between the quality level considered mandatory and its mapping onto the verification pipeline structure (i.e. which stage failures are considered blocking, requiring backout/rollback of the respective changes vs. those for which it's acceptable to just file issues to be fixed down the road)</li>
  <li>the amount of additional effort/resources necessary to identify the culprits for and fix the blocking failures detected by the pipeline executions, to complete the (continuous) integration/delivery/deployment process</li>
  </ul>
  
  <p>In smaller scale projects things are usually simple, but in larger scale ones  these aspects require serious thinking as they can have a huge impact on the overall development process.</p>
  
  <p>I'll use your first question as an example: </p>
  
  <ul>
  <li>if you have a dozen of repositories which combined take less than a minute to checkout/pull - it's probably OK to keep things simple and just pull them all in one step. </li>
  <li>if you have several thousands repositories which may take 1h+ to pull you should probably just pull in each stage what you need for that stage. Or maybe pull them all in the first stage, cache them and re-use that cache in later stages if that helps avoid repeating lengthy pulls. </li>
  </ul>
  
  <p>Whatever makes sense, in your particular project context.</p>
  
Answers: CC BY-SA 4.0
Answers: 2019-06-12T02:53:40.500Z
Answers: 2019-06-12T02:53:40.500Z
Answers: Dan Cornilescu
Answers: <p>Yes, in my perspective stages are mostly meant for a visualisation purpose, and we need to maintain the clear demarcation between stages. My suggestion to go with all the SCM related checkouts in a single stage itself . </p>
  
Answers: CC BY-SA 4.0
Answers: 2019-06-12T01:23:50.937Z
Answers: 2019-06-12T01:23:50.937Z
Answers: Venkat Mani
Body: <p>Should I create single <code>checkout</code> stage or multiple stages like <code>checkout scripts</code>, <code>checkout code</code>, <code>checkout something else</code>?</p>
  
  <p>How should I decide on what piece of logic to put into separate stage?
  Or are stages just a pipeline visualization tool?</p>
  
Tags: <jenkins><jenkins-pipeline>
Title: What is the optimal size of a Jenkins Pipeline Stage?
UserDisplayName: Anton Petrov
ThreadText: # What is the optimal size of a Jenkins Pipeline Stage?
  
  (Asked by: username_0 on 2019-06-06)
  
  Should I create single `checkout` stage or multiple stages like `checkout scripts`, `checkout code`, `checkout something else`?
  
  How should I decide on what piece of logic to put into separate stage?
  Or are stages just a pipeline visualization tool?
  
  **Tags:** jenkins, jenkins-pipeline
  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_1 on 2019-06-06. Score: 3
  
  [Single responsibility principle (SRP)](https://en.wikipedia.org/wiki/Single_responsibility_principle)
  
  > Martin defines a responsibility as a reason to change, and concludes
  > that a class or module should have one, and only one, reason to be
  > changed (i.e. rewritten). As an example, consider a module that
  > compiles and prints a report. Imagine such a module can be changed for
  > two reasons. First, the content of the report could change. Second,
  > the format of the report could change. These two things change for
  > very different causes; one substantive, and one cosmetic. The single
  > responsibility principle says that these two aspects of the problem
  > are really two separate responsibilities, and should therefore be in
  > separate classes or modules. It would be a bad design to couple two
  > things that change for different reasons at different times.
  >
  > The reason it is important to keep a class focused on a single concern
  > is that it makes the class more robust. Continuing with the foregoing
  > example, if there is a change to the report compilation process, there
  > is greater danger that the printing code will break if it is part of
  > the same class.
  
  [5-15 lines](https://softwareengineering.stackexchange.com/a/133406/218283)
  
  > Functions should normally be short, between 5-15 lines is my personal
  > "rule of thumb" when coding in Java or C#. This is a good size for
  > several reasons:
  >
  > * It fits easily on your screen without scrolling
  > * It's about the conceptual size that you can hold in your head
  > * It's meaningful enough to require a function in its own right (as a standalone, meaningful chunk of logic)
  > * A function smaller than 5 lines is a hint that you are perhaps breaking the code up too much (which makes it harder to read /
  >   understand if you need to navigate between functions). Either that or
  >   your're forgetting your special cases / error handling!
  
  **Personal view**
  
  I have the SRP in mind when creating Jenkins stages. This implies that I create a separate stage for checkout, build and publish for example. If a stage becomes too large, i.e. longer than 15 lines, then I create a script and run the script in this stage. One of the benefits is readability, possible to apply unit testing on the script and CI agnostic, i.e. migration to another CI would be possible as well.
  
  [Example](https://jenkins.io/doc/book/pipeline/jenkinsfile/)
  
  > ```
  > pipeline {
  >     agent any
  >
  >     stages {
  >         stage('Build') {
  >             steps {
  >                 echo 'Building..'
  >             }
  >         }
  >         stage('Test') {
  >             steps {
  >                 echo 'Testing..'
  >             }
  >         }
  >         stage('Deploy') {
  >             steps {
  >                 echo 'Deploying....'
  >             }
  >         }
  >     }
  > }
  >
  > ```
  
  
  ---
  
  ### Answer by username_2 on 2019-06-12. Score: 1
  
  In addition to being a pipeline visualization tool, stages also identify portions of the pipeline that could be executed more or less independently from each other, potentially on different workers. Do not underestimate the importance of choosing the right pipeline structure.
  
  Stages can be re-executed if/when needed, for example in case of failures caused by environment issues, without having to re-run the already completed upstream stages.
  
  Stages provide ability for pipeline branching. For example a single build stage can produce artifacts that could be used by multiple downstream test stages than can be executed in parallel, for shorter overall pipeline execution time.
  
  Each stage operation can be supported by pools of workers of different types and sizes.
  
  Stages can be conditionally executed. For example an expensive integration test stage (lengthy and/or with a smaller resource pool than the build stage) could be executed only once in a while rather than for every commit, or only on a release branch and not on the primary development branch.
  
  Since generally the intent is to have the pipeline triggered by each and every commit in the respective branch the size of the pipeline - i.e. the number of stages, their duration and the structure of the pipeline graph (it doesn't necessarily have to be linear) - has a potentially complex relationship with:
  
  * the rate and profile of the commits triggering the pipelines
  * the pool(s) of resources available to support the operation of the pipeline stage(s)
  * the role of the branch, the associated balance between the quality level considered mandatory and its mapping onto the verification pipeline structure (i.e. which stage failures are considered blocking, requiring backout/rollback of the respective changes vs. those for which it's acceptable to just file issues to be fixed down the road)
  * the amount of additional effort/resources necessary to identify the culprits for and fix the blocking failures detected by the pipeline executions, to complete the (continuous) integration/delivery/deployment process
  
  In smaller scale projects things are usually simple, but in larger scale ones these aspects require serious thinking as they can have a huge impact on the overall development process.
  
  I'll use your first question as an example:
  
  * if you have a dozen of repositories which combined take less than a minute to checkout/pull - it's probably OK to keep things simple and just pull them all in one step.
  * if you have several thousands repositories which may take 1h+ to pull you should probably just pull in each stage what you need for that stage. Or maybe pull them all in the first stage, cache them and re-use that cache in later stages if that helps avoid repeating lengthy pulls.
  
  Whatever makes sense, in your particular project context.
  
  
  ---
  
  ### Answer by username_3 on 2019-06-12. Score: 0
  
  Yes, in my perspective stages are mostly meant for a visualisation purpose, and we need to maintain the clear demarcation between stages. My suggestion to go with all the SCM related checkouts in a single stage itself .
  
  
  ---
  
  