Answers: <p>Instead of trying to find a solution in git, I would recommending using the <a href="https://plugins.jenkins.io/lockable-resources/" rel="nofollow noreferrer">Lockable Resources Plugin</a>, which is an official plugin developed by the Jenkins developers.  This plugin implements locks in Jenkins, which are basically the same idea as a <a href="https://en.wikipedia.org/wiki/Lock_(computer_science)" rel="nofollow noreferrer">semaphore or mutex</a>.  Using this plugin, each of your jobs that needs to push to the same branch of the same repo will request a lock when performing the push.  If the lock is available, the job will acquire the lock, push right away, and then release the lock when it is finished pushing.  If the lock is not available, the job will wait until the lock is available (i.e., all other jobs are done pushing) before grabbing it, pushing, and releasing the lock.</p>
  
Answers: CC BY-SA 4.0
Answers: 2020-02-13T15:55:39.850Z
Answers: brilliant! I had no idea this plugin existed and it's exactly what we needed, thanks!
Answers: Alex
Answers: CC BY-SA 4.0
Answers: 2020-02-13T06:04:33.573Z
Answers: 2020-02-13T06:04:33.573Z
Answers: jayhendren
Body: <p>We've got a ton of jobs in our Jenkins that all do git pushes, sometimes simultaneously. Unsurprisingly, if two jobs try to push at the same exact time, one will succeed while the other fails. My knee-jerk reaction is to build some kind of exponential backoff and retry for these git pushes, but I'm pretty unfamiliar with git. Does something like that exist already?</p>
  
  <p>Our commit/push code generally looks like this:</p>
  
  <pre><code>git config --global user.name "username"
  git config --global user.email email@example.com
  git add --ignore-errors -- dir/
  git commit -m "some message"
  git pull --rebase origin
  git push origin master
  </code></pre>
  
  <p>As you can see, the bandaid originally applied to the problem was to put a pull before the push to fix other jobs that pushed while the current one was running. That works for ~80 of cases, but if the two pushes fall within the same &lt;1sec window, there are still failures. </p>
  
  <p>Here's an example of a failure for the last two lines above: (Anything in &lt;> is edited for privacy)</p>
  
  <pre><code>+ git pull --rebase origin
  Warning: Permanently added '&lt;bitbucket.org ip&gt;' (RSA) to the list of known hosts.
  From &lt;bitbucket.org:dir&gt;
     &lt;commit#&gt;..&lt;commit#2&gt;  master     -&gt; origin/master
  First, rewinding head to replay your work on top of it...
  Applying: &lt;commit name&gt;
  + git push origin master
  Warning: Permanently added '&lt;bitbucket.org, ip&gt;' (RSA) to the list of known hosts.
  remote: error: cannot lock ref 'refs/heads/master': is at &lt;commit sha&gt; but expected &lt;different commit sha&gt;        
  To &lt;bitbucket.org:dir&gt;
   ! [remote rejected] master -&gt; master (failed to update ref)
  error: failed to push some refs to 'git@bitbucket.org:&lt;team/dir&gt;'
  </code></pre>
  
  <p>So my question boils down to:</p>
  
  <ol>
  <li>Is there some form of backoff-and-retry built into git?</li>
  <li>If not, how can I build that (assume a shell block in a jenkins freestyle job)?</li>
  </ol>
  
Comments: CC BY-SA 4.0
Comments: 2022-02-03T15:36:17.830Z
Comments: Similar question on stackoverflow: https://stackoverflow.com/q/42743809/873282
Comments: koppor
Tags: <jenkins><git>
Title: exponential backoff for git pushes?
UserDisplayName: Alex
ThreadText: # exponential backoff for git pushes?
  
  (Asked by: username_0 on 2020-02-12)
  
  We've got a ton of jobs in our Jenkins that all do git pushes, sometimes simultaneously. Unsurprisingly, if two jobs try to push at the same exact time, one will succeed while the other fails. My knee-jerk reaction is to build some kind of exponential backoff and retry for these git pushes, but I'm pretty unfamiliar with git. Does something like that exist already?
  
  Our commit/push code generally looks like this:
  
  ```
  git config --global user.name "username"
  git config --global user.email email@example.com
  git add --ignore-errors -- dir/
  git commit -m "some message"
  git pull --rebase origin
  git push origin master
  
  ```
  
  As you can see, the bandaid originally applied to the problem was to put a pull before the push to fix other jobs that pushed while the current one was running. That works for ~80 of cases, but if the two pushes fall within the same <1sec window, there are still failures.
  
  Here's an example of a failure for the last two lines above: (Anything in <> is edited for privacy)
  
  ```
  + git pull --rebase origin
  Warning: Permanently added '<bitbucket.org ip>' (RSA) to the list of known hosts.
  From <bitbucket.org:dir>
     <commit#>..<commit#2>  master     -> origin/master
  First, rewinding head to replay your work on top of it...
  Applying: <commit name>
  + git push origin master
  Warning: Permanently added '<bitbucket.org, ip>' (RSA) to the list of known hosts.
  remote: error: cannot lock ref 'refs/heads/master': is at <commit sha> but expected <different commit sha>        
  To <bitbucket.org:dir>
   ! [remote rejected] master -> master (failed to update ref)
  error: failed to push some refs to 'git@bitbucket.org:<team/dir>'
  
  ```
  
  So my question boils down to:
  
  1. Is there some form of backoff-and-retry built into git?
  2. If not, how can I build that (assume a shell block in a jenkins freestyle job)?
  
  **Tags:** jenkins, git
  
  > **Comments:**
  > **username_1**: Similar question on stackoverflow: https://stackoverflow.com/q/42743809/873282  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-02-13. Score: 2 | Accepted answer
  
  Instead of trying to find a solution in git, I would recommending using the [Lockable Resources Plugin](https://plugins.jenkins.io/lockable-resources/), which is an official plugin developed by the Jenkins developers. This plugin implements locks in Jenkins, which are basically the same idea as a [semaphore or mutex](https://en.wikipedia.org/wiki/Lock_(computer_science)). Using this plugin, each of your jobs that needs to push to the same branch of the same repo will request a lock when performing the push. If the lock is available, the job will acquire the lock, push right away, and then release the lock when it is finished pushing. If the lock is not available, the job will wait until the lock is available (i.e., all other jobs are done pushing) before grabbing it, pushing, and releasing the lock.
  
  > **Comments:**
  > **username_0**: brilliant! I had no idea this plugin existed and it's exactly what we needed, thanks!  
  
  ---
  
  