Answers: <p>I found the answer in this github issue</p>
  <p><a href="https://github.com/docker/cli/issues/1809" rel="nofollow noreferrer">https://github.com/docker/cli/issues/1809</a></p>
  <blockquote>
  <p>if DOCKER_HOST environment variable is set, we ignore the current context setting.</p>
  </blockquote>
  <p>Bitbucket sets this environment variable to the default context meaning I was unable to swap docker context.</p>
  <p>Once I added the command <em>unset DOCKER_HOST</em> the context switch succeeded and docker compose worked as expected.</p>
  
Answers: CC BY-SA 4.0
Answers: 2021-08-12T01:39:42.220Z
Answers: 2021-08-12T01:39:42.220Z
Answers: Jason Redwood
Body: <p>I'm trying to add a deployment step to bitbucket pipelines that works successfully when I simulate locally but fails when executing the steps in bitbucket.</p>
  <p>I have created a simple docker image that contains docker and compose-cli as per <a href="https://raw.githubusercontent.com/docker/compose-cli/main/scripts/install/install_linux.sh" rel="nofollow noreferrer">https://raw.githubusercontent.com/docker/compose-cli/main/scripts/install/install_linux.sh</a></p>
  <p>When I run my &quot;build-env&quot; docker image locally as per bitbuckets debug instructions <a href="https://support.atlassian.com/bitbucket-cloud/docs/debug-pipelines-locally-with-docker/" rel="nofollow noreferrer">https://support.atlassian.com/bitbucket-cloud/docs/debug-pipelines-locally-with-docker/</a> - everything works as expected and I can connect to the AWS ECS docker context and use the &quot;docker compose&quot; commands</p>
  <p>However, when I push the changes to bitbucket and execute the build step in bitbucket pipelines the build fails with &quot;compose&quot; command not found.</p>
  <p>I have inspected env vars and double checked the correct docker CLI is installed and ensured the correct image is being pulled.</p>
  <p>Either the ecs ccontext creation or connection is failing or the wrong docker binary is being called? Or something else i'm not aware of.</p>
  <p>The pipeline yaml</p>
  <pre><code>image: build-env:latest
  
  definitions:
    steps:
      - step: &amp;deploy
          name: Deploy this build
          deployment: test
            - docker context create ecs --from-env deploy
            - docker context use deploy
            - docker version
            - docker compose ps
  </code></pre>
  <p>The dockerfile for the build-env</p>
  <pre><code>FROM ubuntu:20.04
  
  RUN apt-get update
  
  RUN apt-get install -y python3-pip python3-dev \
    &amp;&amp; cd /usr/local/bin \
    &amp;&amp; ln -s /usr/bin/python3 python \
    &amp;&amp; pip3 install --upgrade pip awscli
  
  RUN apt-get install -y curl docker.io
  
  RUN curl -L https://raw.githubusercontent.com/docker/compose-cli/main/scripts/install/install_linux.sh | sh
  
  CMD [&quot;python3&quot;]
  </code></pre>
  <p>The bitbucket output</p>
  <pre><code>docker context create ecs --from-env deploy&lt;1s
  
  + docker context create ecs --from-env deploy
  Successfully created ecs context &quot;deploy&quot;
  docker context use deploy&lt;1s
  
  + docker context use deploy
  deploy
  
  docker version&lt;1s
  
  + docker version
  Cannot connect to the Docker daemon at tcp://localhost:2375. Is the docker daemon running?
  Client:
   Cloud integration: 1.0.17
   Version:           20.10.7
   API version:       1.41
   Go version:        go1.13.8
   Git commit:        20.10.7-0ubuntu1~20.04.1
   Built:             Wed Aug  4 22:52:25 2021
   OS/Arch:           linux/amd64
   Context:           default
   Experimental:      true
  
  docker compose ps&lt;1s
  
  + docker compose ps
  **docker: 'compose' is not a docker command.**
  
  See 'docker --help'
  Build teardown&lt;1s
  </code></pre>
  <p>Very strange that it works locally but fails on bitbuckets servers...  Does anyone know where I'm going wrong???</p>
  <p>The end goal is to use &quot;docker compose up&quot; within the AWS context to trigger the ecs cluster updates. It works perfectly everywhere except for inside bitbuckets environment.</p>
  <p>Thanks in advance....</p>
  <p><strong>Update</strong>
  I have managed to narrow the issue down. Seems the failure is occurring at &quot;docker context use deploy&quot;</p>
  <p>The context is being created and I can inspect and list it, but attempting to use it always fails as seen in the bitbucket pipeline output below.</p>
  <pre><code>docker context use deploy&lt;1s
  
  + docker context use deploy
  deploy
  docker context inspect&lt;1s
  
  + docker context inspect
  [
      {
          &quot;Name&quot;: &quot;default&quot;,
          &quot;Metadata&quot;: {
              &quot;StackOrchestrator&quot;: &quot;swarm&quot;
          },
          &quot;Endpoints&quot;: {
              &quot;docker&quot;: {
                  &quot;Host&quot;: &quot;tcp://localhost:2375&quot;,
                  &quot;SkipTLSVerify&quot;: false
              }
          },
          &quot;TLSMaterial&quot;: {},
          &quot;Storage&quot;: {
              &quot;MetadataPath&quot;: &quot;\u003cIN MEMORY\u003e&quot;,
              &quot;TLSPath&quot;: &quot;\u003cIN MEMORY\u003e&quot;
          }
      }
  ]
  docker context ls&lt;1s
  
  + docker context ls
  NAME                TYPE                DESCRIPTION                               DOCKER ENDPOINT        KUBERNETES ENDPOINT   ORCHESTRATOR
  default *           moby                Current DOCKER_HOST based configuration   tcp://localhost:2375                         swarm
  deploy              ecs                 credentials read from environment                                                      
  </code></pre>
  
Tags: <docker><docker-compose><deployment><continuous-deployment><bitbucket-pipelines>
Title: Unable to use AWS ECS docker context in bitbucket pipelines deployment step
UserDisplayName: Jason Redwood
ThreadText: # Unable to use AWS ECS docker context in bitbucket pipelines deployment step
  
  (Asked by: username_0 on 2021-08-11)
  
  I'm trying to add a deployment step to bitbucket pipelines that works successfully when I simulate locally but fails when executing the steps in bitbucket.
  
  I have created a simple docker image that contains docker and compose-cli as per <https://raw.githubusercontent.com/docker/compose-cli/main/scripts/install/install_linux.sh>
  
  When I run my "build-env" docker image locally as per bitbuckets debug instructions <https://support.atlassian.com/bitbucket-cloud/docs/debug-pipelines-locally-with-docker/> - everything works as expected and I can connect to the AWS ECS docker context and use the "docker compose" commands
  
  However, when I push the changes to bitbucket and execute the build step in bitbucket pipelines the build fails with "compose" command not found.
  
  I have inspected env vars and double checked the correct docker CLI is installed and ensured the correct image is being pulled.
  
  Either the ecs ccontext creation or connection is failing or the wrong docker binary is being called? Or something else i'm not aware of.
  
  The pipeline yaml
  
  ```
  image: build-env:latest
  
  definitions:
    steps:
      - step: &deploy
          name: Deploy this build
          deployment: test
            - docker context create ecs --from-env deploy
            - docker context use deploy
            - docker version
            - docker compose ps
  
  ```
  
  The dockerfile for the build-env
  
  ```
  FROM ubuntu:20.04
  
  RUN apt-get update
  
  RUN apt-get install -y python3-pip python3-dev \
    && cd /usr/local/bin \
    && ln -s /usr/bin/python3 python \
    && pip3 install --upgrade pip awscli
  
  RUN apt-get install -y curl docker.io
  
  RUN curl -L https://raw.githubusercontent.com/docker/compose-cli/main/scripts/install/install_linux.sh | sh
  
  CMD ["python3"]
  
  ```
  
  The bitbucket output
  
  ```
  docker context create ecs --from-env deploy<1s
  
  + docker context create ecs --from-env deploy
  Successfully created ecs context "deploy"
  docker context use deploy<1s
  
  + docker context use deploy
  deploy
  
  docker version<1s
  
  + docker version
  Cannot connect to the Docker daemon at tcp://localhost:2375. Is the docker daemon running?
  Client:
   Cloud integration: 1.0.17
   Version:           20.10.7
   API version:       1.41
   Go version:        go1.13.8
   Git commit:        20.10.7-0ubuntu1~20.04.1
   Built:             Wed Aug  4 22:52:25 2021
   OS/Arch:           linux/amd64
   Context:           default
   Experimental:      true
  
  docker compose ps<1s
  
  + docker compose ps
  **docker: 'compose' is not a docker command.**
  
  See 'docker --help'
  Build teardown<1s
  
  ```
  
  Very strange that it works locally but fails on bitbuckets servers... Does anyone know where I'm going wrong???
  
  The end goal is to use "docker compose up" within the AWS context to trigger the ecs cluster updates. It works perfectly everywhere except for inside bitbuckets environment.
  
  Thanks in advance....
  
  **Update**
  I have managed to narrow the issue down. Seems the failure is occurring at "docker context use deploy"
  
  The context is being created and I can inspect and list it, but attempting to use it always fails as seen in the bitbucket pipeline output below.
  
  ```
  docker context use deploy<1s
  
  + docker context use deploy
  deploy
  docker context inspect<1s
  
  + docker context inspect
  [
      {
          "Name": "default",
          "Metadata": {
              "StackOrchestrator": "swarm"
          },
          "Endpoints": {
              "docker": {
                  "Host": "tcp://localhost:2375",
                  "SkipTLSVerify": false
              }
          },
          "TLSMaterial": {},
          "Storage": {
              "MetadataPath": "\u003cIN MEMORY\u003e",
              "TLSPath": "\u003cIN MEMORY\u003e"
          }
      }
  ]
  docker context ls<1s
  
  + docker context ls
  NAME                TYPE                DESCRIPTION                               DOCKER ENDPOINT        KUBERNETES ENDPOINT   ORCHESTRATOR
  default *           moby                Current DOCKER_HOST based configuration   tcp://localhost:2375                         swarm
  deploy              ecs                 credentials read from environment                                                      
  
  ```
  
  **Tags:** docker, docker-compose, deployment, continuous-deployment, bitbucket-pipelines
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2021-08-12. Score: 2
  
  I found the answer in this github issue
  
  <https://github.com/docker/cli/issues/1809>
  
  > if DOCKER\_HOST environment variable is set, we ignore the current context setting.
  
  Bitbucket sets this environment variable to the default context meaning I was unable to swap docker context.
  
  Once I added the command *unset DOCKER\_HOST* the context switch succeeded and docker compose worked as expected.
  
  
  ---
  
  