Answers: <p>Well this seems to be a docker issue. It’s keeping a whole lot of dangling volumes.</p>
  
  <p>check if you have some :</p>
  
  <pre><code>docker volume ls -qf dangling=true
  </code></pre>
  
  <p>Get rid of them with :</p>
  
  <pre><code>docker volume rm $(docker volume ls -qf dangling=true)
  </code></pre>
  
Answers: CC BY-SA 4.0
Answers: 2019-04-04T11:20:27.433Z
Answers: 2019-04-04T12:04:16.870Z
Answers: 2019-04-04T12:04:16.870Z
Answers: Armel Larcier
Body: <p>I'm using gitlab and gitlab-ci along with gitlab-runner.</p>
  
  <p>Gitlab-runner is choking on my build jobs for a day now and I can't find a way to solve the problem.</p>
  
  <p>The problem appears when the <code>npm install</code> command runs :</p>
  
  <pre><code>npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/array-initial-dd86c469/.jshintrc'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/color-support-612f9677/index.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/clone-94e64b37/clone.iml'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/async-each-23f1cf68/README.md'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/extend-b499a705/.travis.yml'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/function-bind-b92d92f1/implementation.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/fs.realpath-89c6342a/old.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/balanced-match-601d32f6/LICENSE.md'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/default-resolution-1ee87648/index.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/archy-6b1fc3a7/test/beep.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/array-initial-dd86c469/LICENSE-MIT'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/core-util-is-c201cb52/lib/util.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/concat-map-071c084f/example/map.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/buffer-equal-a9e86a30/example/eq.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/gulp-02a15240/package.json'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/flagged-respawn-ed938be5/lib/is-v8flags.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/extend-b499a705/CHANGELOG.md'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/function-bind-b92d92f1/index.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/default-resolution-1ee87648/node-version.js'
  Unhandled rejection Error: ENOSPC: no space left on device, open '/root/.npm/_cacache/index-v5/48/51/88ad48f396d262bcea9e42159a40a93446826c57af0b1c54d2699aca3607'
  Unhandled rejection Error: ENOSPC: no space left on device, mkdir '/root/.npm/_cacache/index-v5/de/70'
  Unhandled rejection Error: ENOSPC: no space left on device, mkdir '/root/.npm/_cacache/index-v5/de/70'
  Unhandled rejection Error: ENOSPC: no space left on device, mkdir '/root/.npm/_cacache/index-v5/2f/98'
  Unhandled rejection Error: ENOSPC: no space left on device, mkdir '/root/.npm/_cacache/index-v5/5d/36'
  Unhandled rejection Error: ENOSPC: no space left on device, mkdir '/root/.npm/_cacache/index-v5/2d/fa'
  Unhandled rejection Error: ENOSPC: no space left on device, mkdir '/root/.npm/_cacache/index-v5/f8/8f'
  Unhandled rejection Error: ENOSPC: no space left on device, mkdir '/root/.npm/_cacache/index-v5/7d/c9'
  Unhandled rejection Error: ENOSPC: no space left on device, mkdir '/root/.npm/_cacache/index-v5/1e/e2'
  Unhandled rejection Error: ENOSPC: no space left on device, mkdir '/root/.npm/_cacache/index-v5/b8/3c'
  </code></pre>
  
  <p>Gitlab-runner is running in a docker container on a photon-os machine.</p>
  
  <p>The cache directory of the runner itself (<code>/builds/sites</code>) is only 60M.</p>
  
  <p>Here is the result of <code>df -h</code> on the host (photon-os)</p>
  
  <pre><code>Filesystem      Size  Used Avail Use% Mounted on
  overlay          16G   13G  2.0G  87% /
  tmpfs            64M     0   64M   0% /dev
  tmpfs          1000M     0 1000M   0% /sys/fs/cgroup
  shm              64M     0   64M   0% /dev/shm
  /dev/root        16G   13G  2.0G  87% /etc/gitlab-runner
  tmpfs          1000M  652K  999M   1% /run/docker.sock
  tmpfs          1000M     0 1000M   0% /proc/acpi
  tmpfs          1000M     0 1000M   0% /sys/firmware
  </code></pre>
  
  <p>and here, df -hi</p>
  
  <pre><code>Filesystem     Inodes IUsed IFree IUse% Mounted on
  overlay          1.0M 1021K  3.8K  100% /
  tmpfs            250K    15  250K    1% /dev
  tmpfs            250K    16  250K    1% /sys/fs/cgroup
  shm              250K     1  250K    1% /dev/shm
  /dev/root        1.0M 1021K  3.8K  100% /etc/gitlab-runner
  tmpfs            250K   470  250K    1% /run/docker.sock
  tmpfs            250K     1  250K    1% /proc/acpi
  tmpfs            250K     1  250K    1% /sys/firmware
  </code></pre>
  
Comments: CC BY-SA 4.0
Comments: 2019-04-04T10:57:17.600Z
Comments: Is by any chance `/` a **btrfs** filesystem?
Comments: Dan Cornilescu
Comments: CC BY-SA 4.0
Comments: 2019-04-04T11:09:01.873Z
Comments: no idea, I never heard of this. How can I know ?
Comments: Armel Larcier
Comments: CC BY-SA 4.0
Comments: 2019-04-04T11:12:11.523Z
Comments: check its entry in `/etc/fstab ` or the output of the `mount` cmd (as root)
Comments: Dan Cornilescu
Comments: CC BY-SA 4.0
Comments: 2019-04-04T11:15:44.857Z
Comments: You have too much files in your runner, there's no more inode free: `/dev/root        1.0M 1021K  3.8K  100% /etc/gitlab-runner`
Comments: Tensibai
Comments: CC BY-SA 4.0
Comments: 2019-04-04T11:16:27.937Z
Comments: Wow I just saw that docker was keeping a whole lot of dangling volumes. Got rid of them and I’m good now! Thanks anyway for the input!
Comments: Armel Larcier
Comments: CC BY-SA 4.0
Comments: 2020-06-25T20:11:21.393Z
Comments: Please tell me exactly how you got gitlab runner running in a docker container on a photon-os machine. I am trying to do that now and am having no luck.
Comments: Shōgun8
Comments: CC BY-SA 4.0
Comments: 2020-06-26T09:23:41.930Z
Comments: Just followed the docs https://docs.gitlab.com/runner/install/docker.html
Comments: Armel Larcier
Comments: CC BY-SA 4.0
Comments: 2020-06-29T20:30:04.330Z
Comments: The problem with that tutorial that is that it requires loading the docker socket via '''-v /var/run/docker.sock:/var/run/docker.sock''' and that is not possible with photon-OS. So can yo please tell me exactly how you did it? Or are you not using the 'Docker Executor'?
Comments: Shōgun8
Tags: <gitlab-ci-runner>
Title: No space left on device when running job on photon-os
UserDisplayName: Armel Larcier
ThreadText: # No space left on device when running job on photon-os
  
  (Asked by: username_0 on 2019-04-04)
  
  I'm using gitlab and gitlab-ci along with gitlab-runner.
  
  Gitlab-runner is choking on my build jobs for a day now and I can't find a way to solve the problem.
  
  The problem appears when the `npm install` command runs :
  
  ```
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/array-initial-dd86c469/.jshintrc'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/color-support-612f9677/index.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/clone-94e64b37/clone.iml'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/async-each-23f1cf68/README.md'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/extend-b499a705/.travis.yml'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/function-bind-b92d92f1/implementation.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/fs.realpath-89c6342a/old.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/balanced-match-601d32f6/LICENSE.md'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/default-resolution-1ee87648/index.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/archy-6b1fc3a7/test/beep.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/array-initial-dd86c469/LICENSE-MIT'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/core-util-is-c201cb52/lib/util.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/concat-map-071c084f/example/map.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/buffer-equal-a9e86a30/example/eq.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/gulp-02a15240/package.json'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/flagged-respawn-ed938be5/lib/is-v8flags.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/extend-b499a705/CHANGELOG.md'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/function-bind-b92d92f1/index.js'
  npm WARN tar ENOSPC: no space left on device, open '/builds/sites/mysite/.npmcache/lib/node_modules/.staging/default-resolution-1ee87648/node-version.js'
  Unhandled rejection Error: ENOSPC: no space left on device, open '/root/.npm/_cacache/index-v5/48/51/88ad48f396d262bcea9e42159a40a93446826c57af0b1c54d2699aca3607'
  Unhandled rejection Error: ENOSPC: no space left on device, mkdir '/root/.npm/_cacache/index-v5/de/70'
  Unhandled rejection Error: ENOSPC: no space left on device, mkdir '/root/.npm/_cacache/index-v5/de/70'
  Unhandled rejection Error: ENOSPC: no space left on device, mkdir '/root/.npm/_cacache/index-v5/2f/98'
  Unhandled rejection Error: ENOSPC: no space left on device, mkdir '/root/.npm/_cacache/index-v5/5d/36'
  Unhandled rejection Error: ENOSPC: no space left on device, mkdir '/root/.npm/_cacache/index-v5/2d/fa'
  Unhandled rejection Error: ENOSPC: no space left on device, mkdir '/root/.npm/_cacache/index-v5/f8/8f'
  Unhandled rejection Error: ENOSPC: no space left on device, mkdir '/root/.npm/_cacache/index-v5/7d/c9'
  Unhandled rejection Error: ENOSPC: no space left on device, mkdir '/root/.npm/_cacache/index-v5/1e/e2'
  Unhandled rejection Error: ENOSPC: no space left on device, mkdir '/root/.npm/_cacache/index-v5/b8/3c'
  
  ```
  
  Gitlab-runner is running in a docker container on a photon-os machine.
  
  The cache directory of the runner itself (`/builds/sites`) is only 60M.
  
  Here is the result of `df -h` on the host (photon-os)
  
  ```
  Filesystem      Size  Used Avail Use% Mounted on
  overlay          16G   13G  2.0G  87% /
  tmpfs            64M     0   64M   0% /dev
  tmpfs          1000M     0 1000M   0% /sys/fs/cgroup
  shm              64M     0   64M   0% /dev/shm
  /dev/root        16G   13G  2.0G  87% /etc/gitlab-runner
  tmpfs          1000M  652K  999M   1% /run/docker.sock
  tmpfs          1000M     0 1000M   0% /proc/acpi
  tmpfs          1000M     0 1000M   0% /sys/firmware
  
  ```
  
  and here, df -hi
  
  ```
  Filesystem     Inodes IUsed IFree IUse% Mounted on
  overlay          1.0M 1021K  3.8K  100% /
  tmpfs            250K    15  250K    1% /dev
  tmpfs            250K    16  250K    1% /sys/fs/cgroup
  shm              250K     1  250K    1% /dev/shm
  /dev/root        1.0M 1021K  3.8K  100% /etc/gitlab-runner
  tmpfs            250K   470  250K    1% /run/docker.sock
  tmpfs            250K     1  250K    1% /proc/acpi
  tmpfs            250K     1  250K    1% /sys/firmware
  
  ```
  
  **Tags:** gitlab-ci-runner
  
  > **Comments:**
  > **username_1**: Is by any chance `/` a **btrfs** filesystem?  
  > **username_0**: no idea, I never heard of this. How can I know ?  
  > **username_1**: check its entry in `/etc/fstab ` or the output of the `mount` cmd (as root)  
  > **username_2**: You have too much files in your runner, there's no more inode free: `/dev/root        1.0M 1021K  3.8K  100% /etc/gitlab-runner`  
  > **username_0**: Wow I just saw that docker was keeping a whole lot of dangling volumes. Got rid of them and I’m good now! Thanks anyway for the input!  
  > **username_3**: Please tell me exactly how you got gitlab runner running in a docker container on a photon-os machine. I am trying to do that now and am having no luck.  
  > **username_0**: Just followed the docs https://docs.gitlab.com/runner/install/docker.html  
  > **username_3**: The problem with that tutorial that is that it requires loading the docker socket via '''-v /var/run/docker.sock:/var/run/docker.sock''' and that is not possible with photon-OS. So can yo please tell me exactly how you did it? Or are you not using the 'Docker Executor'?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2019-04-04. Score: 2 | Accepted answer
  
  Well this seems to be a docker issue. It’s keeping a whole lot of dangling volumes.
  
  check if you have some :
  
  ```
  docker volume ls -qf dangling=true
  
  ```
  
  Get rid of them with :
  
  ```
  docker volume rm $(docker volume ls -qf dangling=true)
  
  ```
  
  
  ---
  
  