Answers: <p>In my case the issue was that QEMU wasn't installed and thus unavailable for emulation, apparently buildkit can get through a large amount of steps before needing emulation.</p>
  <p>I added an init container to the pod template which installs QEMU emulators as required:</p>
  <pre><code>spec:
      initContainers:
      - name: prepare
        image: tonistiigi/binfmt:latest
        args: [&quot;--install&quot;, &quot;arm64&quot;]
        securityContext:
            privileged: true
  </code></pre>
  
Answers: CC BY-SA 4.0
Answers: 2022-03-16T12:52:37.827Z
Answers: 2022-03-16T12:52:37.827Z
Answers: Kristoffer Sall-Storgaard
Body: <p>A few of our developers recently got some M1 machines, I want to provide them with ARM compatible images and have set my sights on Buildkit.</p>
  <p>I set up a podTemplate in Jenkins and run the build with this command</p>
  <pre><code>buildctl build --opt platform=linux/arm64 --frontend=dockerfile.v0 --local context=. --opt build-arg:HTTP_PROXY=http://proxy-here:8080 --opt build-arg:HTTPS_PROXY=http://proxy-here:8080 --local dockerfile=. --output type=image,push=true,\\&quot;name=\$BUILD_TAG,\$SHORT_TAG,\$MINOR_TAG,\$FULL_TAG\\&quot;
  </code></pre>
  <p>When running for <code>linux/amd64</code> everything goes as planned, however <code>linux/arm64</code> will fail with a variety of errors depending on the Dockerfile, for instance:</p>
  <pre><code>RUN update-ca-certificates:
  # Error while loading /usr/bin/run-parts: No such file or directory
  </code></pre>
  <p>I have checked that run-parts exits and that it is linked to to musl with the right architecture.</p>
  <pre><code>RUN which run-parts
  # /bin/run-parts
  
  
  RUN ldd `which run-parts`
  # /lib/ld-musl-aarch64.so.1 (0x5500000000)
  # libc.musl-aarch64.so.1 =&gt; /lib/ld-musl-aarch64.so.1 (0x5500000000)
  
  RUN file `which run-parts`
  # /bin/run-parts: symbolic link to /bin/busybox
  
  RUN file `which busybox`
  # /bin/busybox: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-aarch64.so.1, stripped
  
  RUN ldd `which busybox`
  # /lib/ld-musl-aarch64.so.1 (0x5500000000)
  # libc.musl-aarch64.so.1 =&gt; /lib/ld-musl-aarch64.so.1 (0x5500000000)
  
  RUN file `which update-ca-certificates`
  # /usr/sbin/update-ca-certificates: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-aarch64.so.1, stripped
  </code></pre>
  <p>The error seems to indicate that it is not running with the correct <code>$PATH</code>, but then again, <code>which</code> seems to have no issue finding the commands, <code>/lib/ld-musl-aarch64.so.1</code> also definitely exitsts.</p>
  <p>The error will happen to all sorts of otherwise pedestrian commands <code>ls</code>, <code>addgroup</code> (but not <code>deluser</code>).</p>
  <p>I have tried linking files from <code>/bin/</code> into <code>/usr/bin/</code> but that quickly becomes a game of whack a mole and seems too much of a hack for me.</p>
  <p>The base image for the container is <code>php:7.4.28-fpm-alpine</code></p>
  <p>My Google-fu completely fails me here, what am I doing wrong?</p>
  
Tags: <dockerfile><alpine>
Title: Buildkit failing arm builds with missing binaries even though they are in $PATH
UserDisplayName: Kristoffer Sall-Storgaard
ThreadText: # Buildkit failing arm builds with missing binaries even though they are in $PATH
  
  (Asked by: username_0 on 2022-03-15)
  
  A few of our developers recently got some M1 machines, I want to provide them with ARM compatible images and have set my sights on Buildkit.
  
  I set up a podTemplate in Jenkins and run the build with this command
  
  ```
  buildctl build --opt platform=linux/arm64 --frontend=dockerfile.v0 --local context=. --opt build-arg:HTTP_PROXY=http://proxy-here:8080 --opt build-arg:HTTPS_PROXY=http://proxy-here:8080 --local dockerfile=. --output type=image,push=true,\\"name=\$BUILD_TAG,\$SHORT_TAG,\$MINOR_TAG,\$FULL_TAG\\"
  
  ```
  
  When running for `linux/amd64` everything goes as planned, however `linux/arm64` will fail with a variety of errors depending on the Dockerfile, for instance:
  
  ```
  RUN update-ca-certificates:
  # Error while loading /usr/bin/run-parts: No such file or directory
  
  ```
  
  I have checked that run-parts exits and that it is linked to to musl with the right architecture.
  
  ```
  RUN which run-parts
  # /bin/run-parts
  
  RUN ldd `which run-parts`
  # /lib/ld-musl-aarch64.so.1 (0x5500000000)
  # libc.musl-aarch64.so.1 => /lib/ld-musl-aarch64.so.1 (0x5500000000)
  
  RUN file `which run-parts`
  # /bin/run-parts: symbolic link to /bin/busybox
  
  RUN file `which busybox`
  # /bin/busybox: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-aarch64.so.1, stripped
  
  RUN ldd `which busybox`
  # /lib/ld-musl-aarch64.so.1 (0x5500000000)
  # libc.musl-aarch64.so.1 => /lib/ld-musl-aarch64.so.1 (0x5500000000)
  
  RUN file `which update-ca-certificates`
  # /usr/sbin/update-ca-certificates: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-aarch64.so.1, stripped
  
  ```
  
  The error seems to indicate that it is not running with the correct `$PATH`, but then again, `which` seems to have no issue finding the commands, `/lib/ld-musl-aarch64.so.1` also definitely exitsts.
  
  The error will happen to all sorts of otherwise pedestrian commands `ls`, `addgroup` (but not `deluser`).
  
  I have tried linking files from `/bin/` into `/usr/bin/` but that quickly becomes a game of whack a mole and seems too much of a hack for me.
  
  The base image for the container is `php:7.4.28-fpm-alpine`
  
  My Google-fu completely fails me here, what am I doing wrong?
  
  **Tags:** dockerfile, alpine
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2022-03-16. Score: 0 | Accepted answer
  
  In my case the issue was that QEMU wasn't installed and thus unavailable for emulation, apparently buildkit can get through a large amount of steps before needing emulation.
  
  I added an init container to the pod template which installs QEMU emulators as required:
  
  ```
  spec:
      initContainers:
      - name: prepare
        image: tonistiigi/binfmt:latest
        args: ["--install", "arm64"]
        securityContext:
            privileged: true
  
  ```
  
  
  ---
  
  