Answers: <p>The reason you can't use <code>dotenv</code> variables in the <code>rules</code> clause(s) is that <code>rules</code> are evaluated before the pipeline even starts executing.</p>
  <p>If you have a <code>rules</code> clause like</p>
  <pre class="lang-yml prettyprint-override"><code>rules:
    - if: $CI_COMMIT_TAG === 'v1.0.0'
    when: never
    - when: always
  </code></pre>
  <p>Gitlab CI will check the following things:</p>
  <ol>
  <li>Is the pipeline a <code>Tag Pipeline</code> (from the <code>CI_PIPELINE_SOURCE</code> variable)? If no, <code>CI_COMMIT_TAG</code> won't exist, so the clause doesn't exist, and <code>when:always</code> is the only rule. Add the job to the pipeline.</li>
  <li>If the pipeline is a tag pipeline, is the tag === 'v1.0.0'? Then use <code>when:never</code>. Don't add the job to the pipeline.</li>
  <li>Tag pipeline and the tag !== 'v1.0.0', use <code>when:never</code> and add the job to the pipeline.</li>
  </ol>
  <p>All these things happen before the pipeline is in the <code>created</code> state, so therefore it cannot depend on a variable that is created once the pipeline starts running.</p>
  <p>Knowing this, the way I would solve the problem is to mark the job in Stage B as a manual job like it already is, but when you want to run it automatically, call the Gitlab Jobs API from the Stage A job to run the Stage B job.</p>
  <p>This will use a couple of API Calls:</p>
  <ol>
  <li>First, we need to get the jobs for this particular Pipeline:</li>
  </ol>
  <pre class="lang-bash prettyprint-override"><code>curl --header &quot;PRIVATE-TOKEN: &lt;your_access_token&gt;&quot; &quot;https://gitlab.example.com/api/v4/projects/$CI_PROJECT_ID/pipelines/$CI_PIPELINE_IID/jobs?scope=manual&quot;
  </code></pre>
  <p>This will return all the <code>manual</code> jobs from the pipeline. If you have more than one, you can retrieve the correct one by looking at the value for <code>stage</code> and <code>name</code> from the response. Here's an example from the docs:</p>
  <pre class="lang-json prettyprint-override"><code>[
    {
      &quot;commit&quot;: {
        &quot;author_email&quot;: &quot;admin@example.com&quot;,
        &quot;author_name&quot;: &quot;Administrator&quot;,
        &quot;created_at&quot;: &quot;2015-12-24T16:51:14.000+01:00&quot;,
        &quot;id&quot;: &quot;0ff3ae198f8601a285adcf5c0fff204ee6fba5fd&quot;,
        &quot;message&quot;: &quot;Test the CI integration.&quot;,
        &quot;short_id&quot;: &quot;0ff3ae19&quot;,
        &quot;title&quot;: &quot;Test the CI integration.&quot;
      },
      &quot;coverage&quot;: null,
      &quot;allow_failure&quot;: false,
      &quot;created_at&quot;: &quot;2015-12-24T15:51:21.727Z&quot;,
      &quot;started_at&quot;: &quot;2015-12-24T17:54:24.729Z&quot;,
      &quot;finished_at&quot;: &quot;2015-12-24T17:54:24.921Z&quot;,
      &quot;duration&quot;: 0.192,
      &quot;queued_duration&quot;: 0.023,
      &quot;artifacts_expire_at&quot;: &quot;2016-01-23T17:54:24.921Z&quot;,
      &quot;tag_list&quot;: [
        &quot;docker runner&quot;, &quot;ubuntu18&quot;
      ],
      &quot;id&quot;: 6,
      &quot;name&quot;: &quot;rspec:other&quot;,
      &quot;pipeline&quot;: {
        &quot;id&quot;: 6,
        &quot;project_id&quot;: 1,
        &quot;ref&quot;: &quot;main&quot;,
        &quot;sha&quot;: &quot;0ff3ae198f8601a285adcf5c0fff204ee6fba5fd&quot;,
        &quot;status&quot;: &quot;pending&quot;
      },
      &quot;ref&quot;: &quot;main&quot;,
      &quot;artifacts&quot;: [],
      &quot;runner&quot;: null,
      &quot;stage&quot;: &quot;test&quot;,
      &quot;status&quot;: &quot;failed&quot;,
      &quot;tag&quot;: false,
      &quot;web_url&quot;: &quot;https://example.com/foo/bar/-/jobs/6&quot;,
      &quot;user&quot;: {
        &quot;id&quot;: 1,
        &quot;name&quot;: &quot;Administrator&quot;,
        &quot;username&quot;: &quot;root&quot;,
        &quot;state&quot;: &quot;active&quot;,
        &quot;avatar_url&quot;: &quot;http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&amp;d=identicon&quot;,
        &quot;web_url&quot;: &quot;http://gitlab.dev/root&quot;,
        &quot;created_at&quot;: &quot;2015-12-21T13:14:24.077Z&quot;,
        &quot;bio&quot;: null,
        &quot;location&quot;: null,
        &quot;public_email&quot;: &quot;&quot;,
        &quot;skype&quot;: &quot;&quot;,
        &quot;linkedin&quot;: &quot;&quot;,
        &quot;twitter&quot;: &quot;&quot;,
        &quot;website_url&quot;: &quot;&quot;,
        &quot;organization&quot;: &quot;&quot;
      }
    }
  ]
  </code></pre>
  <p>Once we have the Job ID, we can call the Play Job API:</p>
  <pre class="lang-bash prettyprint-override"><code>curl --request POST --header &quot;PRIVATE-TOKEN: &lt;your_access_token&gt;&quot; &quot;https://gitlab.example.com/api/v4/projects/$CI_PROJECT_ID/jobs/$ID_OF_STAGE_B_JOB/play&quot;
  </code></pre>
  <p>Note: we don't want to use the Predefined Variable <code>$CI_JOB_ID</code> for the Play Job API since that will be the ID of the Stage A job, not the Stage B job.</p>
  <p>Here's the <a href="https://docs.gitlab.com/ee/api/jobs.html#list-pipeline-jobs" rel="nofollow noreferrer">Pipeline Jobs API</a>, and here's the <a href="https://docs.gitlab.com/ee/api/jobs.html#play-a-job" rel="nofollow noreferrer">Play Job API</a> for more information and query parameters.</p>
  
Answers: CC BY-SA 4.0
Answers: 2021-08-26T17:43:22.470Z
Answers: 2022-03-21T16:20:21.773Z
Answers: 2022-03-21T16:20:21.773Z
Answers: Adam Marshall
Body: <p>I need to pass info from stage A to stage B so I can change stage B dynamically.</p>
  <p>Details:</p>
  <ul>
  <li>I have a stage (A) where I validate a terraform plan and check if
  it's going to destroy resources.</li>
  <li>I have another stage (B), which applies the terraform plan, which I want to be triggered manually if stage A detected that resources are going to be destroyed.</li>
  <li>When nothing is being destroyed the terraform apply stage (B) should run automatically</li>
  </ul>
  <p>How can I pass this information from stage A to B?</p>
  <p>Making an artifact in A and using &quot;rules:exist&quot; in stage B is not possible, <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/215100" rel="nofollow noreferrer">issue</a></p>
  <p>And if I understand the <a href="https://docs.gitlab.com/ee/ci/variables/#pass-an-environment-variable-to-another-job" rel="nofollow noreferrer">docs</a> correctly it's not possible to use a &quot;dotenv&quot; var in the &quot;rules:if&quot; section of stage B.</p>
  <p>It seems like Gitlab doesn't want you to change the pipeline definition dynamically.
  Maybe there is a practical solution that I'm not aware of to solve this usecase?</p>
  
Tags: <terraform><gitlab><pipeline>
Title: Gitlab CI: change pipeline definition dynamically
UserDisplayName: Pablo
ThreadText: # Gitlab CI: change pipeline definition dynamically
  
  (Asked by: username_0 on 2021-08-17)
  
  I need to pass info from stage A to stage B so I can change stage B dynamically.
  
  Details:
  
  * I have a stage (A) where I validate a terraform plan and check if
    it's going to destroy resources.
  * I have another stage (B), which applies the terraform plan, which I want to be triggered manually if stage A detected that resources are going to be destroyed.
  * When nothing is being destroyed the terraform apply stage (B) should run automatically
  
  How can I pass this information from stage A to B?
  
  Making an artifact in A and using "rules:exist" in stage B is not possible, [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/215100)
  
  And if I understand the [docs](https://docs.gitlab.com/ee/ci/variables/#pass-an-environment-variable-to-another-job) correctly it's not possible to use a "dotenv" var in the "rules:if" section of stage B.
  
  It seems like Gitlab doesn't want you to change the pipeline definition dynamically.
  Maybe there is a practical solution that I'm not aware of to solve this usecase?
  
  **Tags:** terraform, gitlab, pipeline
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-08-26. Score: 3 | Accepted answer
  
  The reason you can't use `dotenv` variables in the `rules` clause(s) is that `rules` are evaluated before the pipeline even starts executing.
  
  If you have a `rules` clause like
  
  ```yml
  rules:
    - if: $CI_COMMIT_TAG === 'v1.0.0'
    when: never
    - when: always
  
  ```
  
  Gitlab CI will check the following things:
  
  1. Is the pipeline a `Tag Pipeline` (from the `CI_PIPELINE_SOURCE` variable)? If no, `CI_COMMIT_TAG` won't exist, so the clause doesn't exist, and `when:always` is the only rule. Add the job to the pipeline.
  2. If the pipeline is a tag pipeline, is the tag === 'v1.0.0'? Then use `when:never`. Don't add the job to the pipeline.
  3. Tag pipeline and the tag !== 'v1.0.0', use `when:never` and add the job to the pipeline.
  
  All these things happen before the pipeline is in the `created` state, so therefore it cannot depend on a variable that is created once the pipeline starts running.
  
  Knowing this, the way I would solve the problem is to mark the job in Stage B as a manual job like it already is, but when you want to run it automatically, call the Gitlab Jobs API from the Stage A job to run the Stage B job.
  
  This will use a couple of API Calls:
  
  1. First, we need to get the jobs for this particular Pipeline:
  
  ```bash
  curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/$CI_PROJECT_ID/pipelines/$CI_PIPELINE_IID/jobs?scope=manual"
  
  ```
  
  This will return all the `manual` jobs from the pipeline. If you have more than one, you can retrieve the correct one by looking at the value for `stage` and `name` from the response. Here's an example from the docs:
  
  ```json
  [
    {
      "commit": {
        "author_email": "admin@example.com",
        "author_name": "Administrator",
        "created_at": "2015-12-24T16:51:14.000+01:00",
        "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
        "message": "Test the CI integration.",
        "short_id": "0ff3ae19",
        "title": "Test the CI integration."
      },
      "coverage": null,
      "allow_failure": false,
      "created_at": "2015-12-24T15:51:21.727Z",
      "started_at": "2015-12-24T17:54:24.729Z",
      "finished_at": "2015-12-24T17:54:24.921Z",
      "duration": 0.192,
      "queued_duration": 0.023,
      "artifacts_expire_at": "2016-01-23T17:54:24.921Z",
      "tag_list": [
        "docker runner", "ubuntu18"
      ],
      "id": 6,
      "name": "rspec:other",
      "pipeline": {
        "id": 6,
        "project_id": 1,
        "ref": "main",
        "sha": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
        "status": "pending"
      },
      "ref": "main",
      "artifacts": [],
      "runner": null,
      "stage": "test",
      "status": "failed",
      "tag": false,
      "web_url": "https://example.com/foo/bar/-/jobs/6",
      "user": {
        "id": 1,
        "name": "Administrator",
        "username": "root",
        "state": "active",
        "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
        "web_url": "http://gitlab.dev/root",
        "created_at": "2015-12-21T13:14:24.077Z",
        "bio": null,
        "location": null,
        "public_email": "",
        "skype": "",
        "linkedin": "",
        "twitter": "",
        "website_url": "",
        "organization": ""
      }
    }
  ]
  
  ```
  
  Once we have the Job ID, we can call the Play Job API:
  
  ```bash
  curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/$CI_PROJECT_ID/jobs/$ID_OF_STAGE_B_JOB/play"
  
  ```
  
  Note: we don't want to use the Predefined Variable `$CI_JOB_ID` for the Play Job API since that will be the ID of the Stage A job, not the Stage B job.
  
  Here's the [Pipeline Jobs API](https://docs.gitlab.com/ee/api/jobs.html#list-pipeline-jobs), and here's the [Play Job API](https://docs.gitlab.com/ee/api/jobs.html#play-a-job) for more information and query parameters.
  
  
  ---
  
  