Answers: <p>We didn't find the cause, neither a real solution.</p>
  
  <p>There is a workaround though:</p>
  
  <p>Put a pipeline stage with <code>rm -rf *</code> before your <code>scm checkout</code> stage. This solves the problem but I will change the accepted answer to whoever posts the real reason and fix for the problem.</p>
  
  <p><strong>Sidenote:</strong></p>
  
  <p>Keep in mind that you will erase anything in your Jenkins workspace doing that. So be certain that you have everything you need in Git so it comes back with the checkout or get it back some other way.</p>
  
  <p>If you rely on files in your Jenkins workspace from previous runs then this method is not for you!</p>
  
  <p>(Although you might want to think about adapting your Job so files from previous runs are not needed anymore. Every run should contain the files it needs by itself and not rely on other runs.)</p>
  
Answers: CC BY-SA 4.0
Answers: 2020-01-03T17:10:52.230Z
Answers: I've had this happen when doing non-containerized Habitat package builds. In my case, the habitat studio aborted a build, but hadn't unmounted its bind mounts of, e g. `/dev`, `/proc` and friends. I discovered the hard way that a sledgehammer `rm -rf` was a quick way to trash the node. My solution was twofold: first, mount `/var/lib/jenkins` as `tmpfs` (so rebooting could cleanly fix the problem on a per-incident basis), and, later, switch to fully-containerized builds to ensure proper cleanup.
Answers: Michael Mol
Answers: CC BY-SA 4.0
Answers: 2020-01-02T12:30:28.120Z
Answers: 2020-01-02T12:30:28.120Z
Answers: Worp
Answers: <p>I think this <em>might</em> just be a permissions issue. You can test this by manually running <code>git clean -fdx</code> as the user running Jenkins. If it doesn't work, try it again with <code>sudo git clean -fdx</code>. If it works, then it's probably an issue with permissions or ownership on one or more files/directories within the repo.</p>
  
  <p>As an alternative, you can install the <a href="https://plugins.jenkins.io/ws-cleanup" rel="nofollow noreferrer">Workspace Cleanup</a> plugin, which provides the <code>cleanWs()</code> function which is used to clean the workspace directory. There are different options, so consult the <a href="https://jenkins.io/doc/pipeline/steps/ws-cleanup/" rel="nofollow noreferrer">documentation</a> to see what might work best for your environment.</p>
  
Answers: CC BY-SA 4.0
Answers: 2020-01-02T13:56:06.693Z
Answers: 2020-01-02T13:56:06.693Z
Answers: Argyle
Answers: <p>At least on Windows, if a process is running in your <code>bin/</code> directory, or for some other reason it has a file handle locked, you will get this error.</p>
  <p>Like you saw, waiting and retrying is a solution. We automate that in our Jenkinsfile: we run <code>git clean -fdx</code> in a retry-sleep loop:</p>
  <pre class="lang-java prettyprint-override"><code>    int retryAttempt = 0
      retry(3) {
          if (retryAttempt &gt; 0) {
              sleep(2 + 5 * retryAttempt)
          }
          retryAttempt = retryAttempt + 1
  
          run('git clean -xfd')
      }
  </code></pre>
  
Answers: CC BY-SA 4.0
Answers: 2024-09-25T00:27:08.097Z
Answers: 2024-09-25T00:27:08.097Z
Answers: Carl Walsh
Body: <p>I have my Jenkins build configured to "Clean before Checkout" for its git source:</p>
  
  <pre><code>Job Config &gt; Source &gt; git &gt; Behaviours &gt; Clean before checkout
  </code></pre>
  
  <p>This results in</p>
  
  <pre><code>Resetting working tree
  
  git reset --hard # timeout=10
  git clean -fdx # timeout=10
  </code></pre>
  
  <p>Before the job begins.</p>
  
  <p>This errors out with</p>
  
  <pre><code>stderr: warning: failed to remove web/static/node_modules/.../fonts/google-fonts: Directory not empty
  </code></pre>
  
  <p>If I re-run the job (simply replay), I get the same error.</p>
  
  <p>If I re-run it again, git clean works fine and the job finishes with success in the end.</p>
  
  <p>If I start it again it errors out 2 times again until it works at the third time. This is always the case.</p>
  
  <p>What might be going on here?</p>
  
Comments: CC BY-SA 4.0
Comments: 2019-11-02T16:54:45.897Z
Comments: Are by any chance builds recently executed or aborted on that node around the time these events happen? Check for any leftover processes from previous builds (or other processes) still running on that node when `git clean` executes, they may keep file descriptors open or create new files in the directory tree being removed, preventing removal. Even a regular shell with the `cwd` set somewhere within that tree can cause that.
Comments: Dan Cornilescu
Comments: CC BY-SA 4.0
Comments: 2019-11-03T12:02:04.893Z
Comments: Could you try to remove the workspace and start the build again?
Comments: 030
Comments: CC BY-SA 4.0
Comments: 2019-11-04T06:54:39.753Z
Comments: @DanCornilescu That is good input. No processes are running (checking by htop and ps aux), but I can try to restart the Container on which Jenkins is running after each job run to be absolutely certain.
Comments: Worp
Comments: CC BY-SA 4.0
Comments: 2019-11-04T07:02:02.333Z
Comments: @030 A pipeline stage with `rm -rf *` before the stage with `scm checkout` solves the problem in deed. Do you want to put that into an answer? If we can't find the actual explanation I'd accept your answer as it's a helpful workaround.
Comments: Worp
Comments: CC BY-SA 4.0
Comments: 2019-11-04T11:56:50.563Z
Comments: @Worp Nice to hear that it works. Yes please post an answer and let me know once you created it.
Comments: 030
Comments: CC BY-SA 4.0
Comments: 2019-12-20T11:33:00.663Z
Comments: @Worp could you post an answer?
Comments: 030
Tags: <jenkins><jenkins-pipeline><git>
Title: Jenkins Pipeline - Clean before checkout - git clean -fdx "failed to remove: Directory not empty"
UserDisplayName: Worp
ThreadText: # Jenkins Pipeline - Clean before checkout - git clean -fdx "failed to remove: Directory not empty"
  
  (Asked by: username_0 on 2019-10-30)
  
  I have my Jenkins build configured to "Clean before Checkout" for its git source:
  
  ```
  Job Config > Source > git > Behaviours > Clean before checkout
  
  ```
  
  This results in
  
  ```
  Resetting working tree
  
  git reset --hard # timeout=10
  git clean -fdx # timeout=10
  
  ```
  
  Before the job begins.
  
  This errors out with
  
  ```
  stderr: warning: failed to remove web/static/node_modules/.../fonts/google-fonts: Directory not empty
  
  ```
  
  If I re-run the job (simply replay), I get the same error.
  
  If I re-run it again, git clean works fine and the job finishes with success in the end.
  
  If I start it again it errors out 2 times again until it works at the third time. This is always the case.
  
  What might be going on here?
  
  **Tags:** jenkins, jenkins-pipeline, git
  
  > **Comments:**
  > **username_1**: Are by any chance builds recently executed or aborted on that node around the time these events happen? Check for any leftover processes from previous builds (or other processes) still running on that node when `git clean` executes, they may keep file descriptors open or create new files in the directory tree being removed, preventing removal. Even a regular shell with the `cwd` set somewhere within that tree can cause that.  
  > **username_2**: Could you try to remove the workspace and start the build again?  
  > **username_0**: @username_1 That is good input. No processes are running (checking by htop and ps aux), but I can try to restart the Container on which Jenkins is running after each job run to be absolutely certain.  
  > **username_0**: @username_2 A pipeline stage with `rm -rf *` before the stage with `scm checkout` solves the problem in deed. Do you want to put that into an answer? If we can't find the actual explanation I'd accept your answer as it's a helpful workaround.  
  > **username_2**: @username_0 Nice to hear that it works. Yes please post an answer and let me know once you created it.  
  > **username_2**: @username_0 could you post an answer?  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_0 on 2020-01-02. Score: 0 | Accepted answer
  
  We didn't find the cause, neither a real solution.
  
  There is a workaround though:
  
  Put a pipeline stage with `rm -rf *` before your `scm checkout` stage. This solves the problem but I will change the accepted answer to whoever posts the real reason and fix for the problem.
  
  **Sidenote:**
  
  Keep in mind that you will erase anything in your Jenkins workspace doing that. So be certain that you have everything you need in Git so it comes back with the checkout or get it back some other way.
  
  If you rely on files in your Jenkins workspace from previous runs then this method is not for you!
  
  (Although you might want to think about adapting your Job so files from previous runs are not needed anymore. Every run should contain the files it needs by itself and not rely on other runs.)
  
  > **Comments:**
  > **username_3**: I've had this happen when doing non-containerized Habitat package builds. In my case, the habitat studio aborted a build, but hadn't unmounted its bind mounts of, e g. `/dev`, `/proc` and friends. I discovered the hard way that a sledgehammer `rm -rf` was a quick way to trash the node. My solution was twofold: first, mount `/var/lib/jenkins` as `tmpfs` (so rebooting could cleanly fix the problem on a per-incident basis), and, later, switch to fully-containerized builds to ensure proper cleanup.  
  
  ---
  
  ### Answer by username_4 on 2020-01-02. Score: 0
  
  I think this *might* just be a permissions issue. You can test this by manually running `git clean -fdx` as the user running Jenkins. If it doesn't work, try it again with `sudo git clean -fdx`. If it works, then it's probably an issue with permissions or ownership on one or more files/directories within the repo.
  
  As an alternative, you can install the [Workspace Cleanup](https://plugins.jenkins.io/ws-cleanup) plugin, which provides the `cleanWs()` function which is used to clean the workspace directory. There are different options, so consult the [documentation](https://jenkins.io/doc/pipeline/steps/ws-cleanup/) to see what might work best for your environment.
  
  
  ---
  
  ### Answer by username_5 on 2024-09-25. Score: 0
  
  At least on Windows, if a process is running in your `bin/` directory, or for some other reason it has a file handle locked, you will get this error.
  
  Like you saw, waiting and retrying is a solution. We automate that in our Jenkinsfile: we run `git clean -fdx` in a retry-sleep loop:
  
  ```java
      int retryAttempt = 0
      retry(3) {
          if (retryAttempt > 0) {
              sleep(2 + 5 * retryAttempt)
          }
          retryAttempt = retryAttempt + 1
  
          run('git clean -xfd')
      }
  
  ```
  
  
  ---
  
  