Answers: <p>Caching of layers with buildkit in an external registry requires an extra step or two depending on how you want to cache your layers. The easy option is to include a build arg that enables the inline cache:</p>
  
  <pre><code>docker build --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from ${LATEST} -t ${LATEST} -t ${IMAGE_COMMIT_TAG} .
  </code></pre>
  
  <p>Note that the inline cache only caches layers for the target stage that was pushed in the image, so other states in a multi-stage build would need to be built and cached separately, or rebuilt without caching, neither of which is ideal.</p>
  
  <p>You can also cache to a local file, or push the cache to a different registry image rather than inline with the image you pushed. Unfortunately the standard <code>docker build</code> CLI doesn't have access to all the buildkit flags to enable this. Instead, you can install buildkit directly or use buildx which is a CLI plugin for managing buildkit. You can install buildx separately but in current releases it's available with experimental CLI options that can be enabled with <code>export DOCKER_CLI_EXPERIMENTAL=enabled</code>. I believe you'd need to create a container based builder for all of the buildkit options, which can be done with:</p>
  
  <pre><code>docker buildx create --use --driver docker-container --name local ${DOCKER_HOST:-unix:///var/run/docker.sock}
  </code></pre>
  
  <p>Then you should be able to run something like:</p>
  
  <pre><code>docker buildx build --cache-from ${IMG_CACHE} --cache-to ${IMG_CACHE} -t ${LATEST} -t ${IMAGE_COMMIT_TAG} .
  </code></pre>
  
  <p>More documentation on the buildkit caching is available at: <a href="https://github.com/moby/buildkit#export-cache" rel="nofollow noreferrer">https://github.com/moby/buildkit#export-cache</a></p>
  
  <p>More details on buildx is available at: <a href="https://github.com/docker/buildx" rel="nofollow noreferrer">https://github.com/docker/buildx</a></p>
  
Answers: CC BY-SA 4.0
Answers: 2020-04-01T10:58:33.327Z
Answers: HI, @BMitch!
  Sorry for such a late response (I was offline up to now) and thank you for your answer.
  
  I know about `buildx`, but it's not applicable, as I don't have any Docker stuff persistent. I have immutable GL-runner :( and that's why I have to use Docker as service.
  They same reason is about using external cache.
Answers: acd
Answers: CC BY-SA 4.0
Answers: 2020-04-01T11:02:25.693Z
Answers: BTW, when I run the build-stage without `DinD` (on different GL-runner) and any external cache, Docker uses cache from the built images not its own build-cache.
Answers: acd
Answers: CC BY-SA 4.0
Answers: 2020-04-01T11:51:26.267Z
Answers: @acd an immutable runner would not be able to build an image. Perhaps you mean an ephemeral runner, which should not prevent you from running docker commands.
Answers: BMitch
Answers: CC BY-SA 4.0
Answers: 2020-04-01T16:50:33.823Z
Answers: No-no. "Immutable runner" means persistent instance, which can't be changed, that is to setup `buildkitd`, `buildx`, as a Docker extension, etc.. And, Docker is ephemeral, which is Docker-in-Docker, used in `services` of .gitlab-ci.
Answers: acd
Answers: CC BY-SA 4.0
Answers: 2020-04-01T18:02:41.547Z
Answers: @acd I'm not finding much in my search for gitlab immutable runner, perhaps you're referring to a shared runner? Immutable would mean no write access to /var/lib/docker which would mean all image builds would fail because of a read-only filesystem. It's also not clear from the comments that you attempted all three of the options suggested above. Output of the errors would help.
Answers: BMitch
Answers: CC BY-SA 4.0
Answers: 2020-04-01T19:26:03.233Z
Answers: Sorry for confusion. I intentionally used the term "immutable" and gave you the explanation of it, what I meant. Here described how it works: https://docs.gitlab.com/ee/ci/yaml/README.html#services.
Answers: acd
Answers: CC BY-SA 4.0
Answers: 2020-04-01T19:28:22.443Z
Answers: you also asked for logs. Here is:  failed to build LLB: rpc error: code = Canceled desc = grpc: the client connection is closing
Answers: acd
Answers: CC BY-SA 4.0
Answers: 2020-04-01T19:49:48.033Z
Answers: @acd Is that error the same for all three options? What did your yml file look like when you implemented each option? It's best to update your question with what you tried.
Answers: BMitch
Answers: CC BY-SA 4.0
Answers: 2020-03-27T00:07:46.760Z
Answers: 2020-03-27T00:14:09.653Z
Answers: 2020-03-27T00:14:09.653Z
Answers: BMitch
Answers: <p>Though it's not the direct answer to my question, but it's a goal, I was aimed at.</p>
  
  <p>I tested several image build tools and stopped on 'Makisu', for building Docker images. It really works fine for me.</p>
  
  <p>The details are here:
  <a href="https://github.com/uber/makisu" rel="nofollow noreferrer">https://github.com/uber/makisu</a></p>
  
  <p>and here (starting from 'Using Makisu'):
  <a href="https://findwork.dev/blog/6-ways-speed-your-ci/" rel="nofollow noreferrer">https://findwork.dev/blog/6-ways-speed-your-ci/</a></p>
  
Answers: CC BY-SA 4.0
Answers: 2023-07-25T22:02:37.503Z
Answers: Makisu is deprecated as of May 2021. [Kaniko](https://github.com/GoogleContainerTools/kaniko) is a similar tool still being developed. [Here are the GitLab Docs for it](https://docs.gitlab.com/ee/ci/docker/using_kaniko.html)
Answers: Nick
Answers: CC BY-SA 4.0
Answers: 2020-04-21T18:03:52.033Z
Answers: 2020-04-21T18:03:52.033Z
Answers: acd
Body: <p>I use GitLab's pipelines to describe the deployment process. And, as part of it, I build Docker images. For that purpose, I use Docker-in-Docker to build those images (i.e. download the <code>Docker-in-Docker</code> image and do all the Docker-stuff, inside of it).</p>
  
  <p>I also set to use Docker Buildkit features. And one of them is <code>--cache-from</code> option (which decreases build time). Which seems doesn't work with <code>Docker-in-Docker</code> (at least, for me).</p>
  
  <p>Here's the pipeline's config:</p>
  
  <pre><code>stages:
    - build
    - purge
    - deploy
  
  variables:
    DOCKER_HOST: tcp://localhost:2375
    DOCKER_TLS_CERTDIR: ""
    DOCKER_BUILDKIT: 1
    LATEST: ${CI_REGISTRY_IMAGE}/${CI_PROJECT_NAME}:${CI_COMMIT_REF_SLUG}
    IMAGE_COMMIT_TAG: ${CI_REGISTRY_IMAGE}/${CI_PROJECT_NAME}:${CI_COMMIT_SHORT_SHA}
  
  services:
    - docker:dind
  
  build:
      stage: build
      image: docker:stable
      script:
          - docker login -u gitlab-ci-token -p ${CI_BUILD_TOKEN} ${CI_REGISTRY}
          - docker build --cache-from ${LATEST} -t ${LATEST} -t ${IMAGE_COMMIT_TAG} .
          - docker push ${IMAGE_COMMIT_TAG}
          - docker push ${LATEST}
  </code></pre>
  
  <p>The log output shows the <code>BuildKit</code> is engaged, but there's nothing about using of 'CACHE'. And, the result of it -- 'build' stage takes much more time.</p>
  
  <p>Please, your suggestions, ideas?</p>
  
Tags: <docker><gitlab><gitlab-ci-runner><docker-in-docker>
Title: Build Docker images with DinD and BuildKit
UserDisplayName: acd
ThreadText: # Build Docker images with DinD and BuildKit
  
  (Asked by: username_0 on 2020-03-26)
  
  I use GitLab's pipelines to describe the deployment process. And, as part of it, I build Docker images. For that purpose, I use Docker-in-Docker to build those images (i.e. download the `Docker-in-Docker` image and do all the Docker-stuff, inside of it).
  
  I also set to use Docker Buildkit features. And one of them is `--cache-from` option (which decreases build time). Which seems doesn't work with `Docker-in-Docker` (at least, for me).
  
  Here's the pipeline's config:
  
  ```
  stages:
    - build
    - purge
    - deploy
  
  variables:
    DOCKER_HOST: tcp://localhost:2375
    DOCKER_TLS_CERTDIR: ""
    DOCKER_BUILDKIT: 1
    LATEST: ${CI_REGISTRY_IMAGE}/${CI_PROJECT_NAME}:${CI_COMMIT_REF_SLUG}
    IMAGE_COMMIT_TAG: ${CI_REGISTRY_IMAGE}/${CI_PROJECT_NAME}:${CI_COMMIT_SHORT_SHA}
  
  services:
    - docker:dind
  
  build:
      stage: build
      image: docker:stable
      script:
          - docker login -u gitlab-ci-token -p ${CI_BUILD_TOKEN} ${CI_REGISTRY}
          - docker build --cache-from ${LATEST} -t ${LATEST} -t ${IMAGE_COMMIT_TAG} .
          - docker push ${IMAGE_COMMIT_TAG}
          - docker push ${LATEST}
  
  ```
  
  The log output shows the `BuildKit` is engaged, but there's nothing about using of 'CACHE'. And, the result of it -- 'build' stage takes much more time.
  
  Please, your suggestions, ideas?
  
  **Tags:** docker, gitlab, gitlab-ci-runner, docker-in-docker
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2020-03-27. Score: 4
  
  Caching of layers with buildkit in an external registry requires an extra step or two depending on how you want to cache your layers. The easy option is to include a build arg that enables the inline cache:
  
  ```
  docker build --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from ${LATEST} -t ${LATEST} -t ${IMAGE_COMMIT_TAG} .
  
  ```
  
  Note that the inline cache only caches layers for the target stage that was pushed in the image, so other states in a multi-stage build would need to be built and cached separately, or rebuilt without caching, neither of which is ideal.
  
  You can also cache to a local file, or push the cache to a different registry image rather than inline with the image you pushed. Unfortunately the standard `docker build` CLI doesn't have access to all the buildkit flags to enable this. Instead, you can install buildkit directly or use buildx which is a CLI plugin for managing buildkit. You can install buildx separately but in current releases it's available with experimental CLI options that can be enabled with `export DOCKER_CLI_EXPERIMENTAL=enabled`. I believe you'd need to create a container based builder for all of the buildkit options, which can be done with:
  
  ```
  docker buildx create --use --driver docker-container --name local ${DOCKER_HOST:-unix:///var/run/docker.sock}
  
  ```
  
  Then you should be able to run something like:
  
  ```
  docker buildx build --cache-from ${IMG_CACHE} --cache-to ${IMG_CACHE} -t ${LATEST} -t ${IMAGE_COMMIT_TAG} .
  
  ```
  
  More documentation on the buildkit caching is available at: <https://github.com/moby/buildkit#export-cache>
  
  More details on buildx is available at: <https://github.com/docker/buildx>
  
  > **Comments:**
  > **username_0**: HI, @username_1! Sorry for such a late response (I was offline up to now) and thank you for your answer.  I know about `buildx`, but it's not applicable, as I don't have any Docker stuff persistent. I have immutable GL-runner :( and that's why I have to use Docker as service. They same reason is about using external cache.  
  > **username_0**: BTW, when I run the build-stage without `DinD` (on different GL-runner) and any external cache, Docker uses cache from the built images not its own build-cache.  
  > **username_1**: @username_0 an immutable runner would not be able to build an image. Perhaps you mean an ephemeral runner, which should not prevent you from running docker commands.  
  > **username_0**: No-no. "Immutable runner" means persistent instance, which can't be changed, that is to setup `buildkitd`, `buildx`, as a Docker extension, etc.. And, Docker is ephemeral, which is Docker-in-Docker, used in `services` of .gitlab-ci.  
  > **username_1**: @username_0 I'm not finding much in my search for gitlab immutable runner, perhaps you're referring to a shared runner? Immutable would mean no write access to /var/lib/docker which would mean all image builds would fail because of a read-only filesystem. It's also not clear from the comments that you attempted all three of the options suggested above. Output of the errors would help.  
  > **username_0**: Sorry for confusion. I intentionally used the term "immutable" and gave you the explanation of it, what I meant. Here described how it works: https://docs.gitlab.com/ee/ci/yaml/README.html#services.  
  > **username_0**: you also asked for logs. Here is:  failed to build LLB: rpc error: code = Canceled desc = grpc: the client connection is closing  
  > **username_1**: @username_0 Is that error the same for all three options? What did your yml file look like when you implemented each option? It's best to update your question with what you tried.  
  
  ---
  
  ### Answer by username_0 on 2020-04-21. Score: 0
  
  Though it's not the direct answer to my question, but it's a goal, I was aimed at.
  
  I tested several image build tools and stopped on 'Makisu', for building Docker images. It really works fine for me.
  
  The details are here:
  <https://github.com/uber/makisu>
  
  and here (starting from 'Using Makisu'):
  <https://findwork.dev/blog/6-ways-speed-your-ci/>
  
  > **Comments:**
  > **username_2**: Makisu is deprecated as of May 2021. [Kaniko](https://github.com/GoogleContainerTools/kaniko) is a similar tool still being developed. [Here are the GitLab Docs for it](https://docs.gitlab.com/ee/ci/docker/using_kaniko.html)  
  
  ---
  
  