title: Docker connectivity issues (to Azure DevOps Services from self hosted Linux Docker agent)
body: <p>I am looking for some advice on debugging some extremely painful Docker connectivity issues.</p>
  <p>In particular, for an Azure DevOps Services Git repository, I am running a self-hosted (locally) dockerized Linux CI (setup according to <a href="https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/docker?view=azure-devops#linux" rel="nofollow noreferrer">https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/docker?view=azure-devops#linux</a>), which has been working fine for a few months now.</p>
  <p>All this runs on a company network, and since last week the network connection of my docker container became highly unstable:</p>
  <ul>
  <li><p>Specifically it intermittently looses network connection, which is also visible via the logs of the Azure DevOps agent, which then keeps trying to reconnect.</p>
  </li>
  <li><p>This especially happens while downloading Git LFS objects. Enabling extra traces via GIT_TRACE=1 highlights a lot of connection failures and retries:</p>
  <blockquote>
  <p>trace git-lfs: xfer: failed to resume download for &quot;SHA&quot; from byte N: expected status code 206, received 200. Re-downloading from start</p>
  </blockquote>
  </li>
  <li><p>During such a LFS pull / fetch, sometimes the container even stops responding as a <code>docker container list</code> command only responds:</p>
  <blockquote>
  <p>Error response from daemon: i/o timeout</p>
  </blockquote>
  <p>As a result the daemon cannot recover on its own, and needs a manual restart (to get back up the CI).</p>
  </li>
  </ul>
  <p>Also I see remarkable differences in network performance:</p>
  <ul>
  <li>Manually cloning the same Git repository (including LFS objects, all from scratch) in <strong>container instances (created from the same image) on different machines</strong>, takes less than 2mins on my dev laptop machine (connected from home via VPN), while the same operation <strong>easily takes up to 20minutes (!) on containers running two different Win10 machines</strong> (company network, physically located in offices, hence no VPN.</li>
  <li>Clearly this is not about the host network connection itself, since cloning on the same Win10 hosts (company network/offices) outside of the containers takes only 14seconds!</li>
  </ul>
  <p>Hence I am suspecting some network configuration issues (e.g. sth with the Hyper-V vEthernet Adapter? Firewall? Proxy? or whichever other watchdog going astray?), but after three days of debugging, I am not quite sure how to further investigate this issue, as I am running out of ideas and expertise. Any thoughts / advice / hints?</p>
  <p>I should add that LFS configuration options (<a href="https://github.com/git-lfs/git-lfs/blob/master/docs/man/git-lfs-config.5.ronn#transfer-upload--download-settings" rel="nofollow noreferrer">such as lfs.concurrenttransfers and lfs.basictransfersonly</a>) did not really help, similarly for <a href="https://git-scm.com/docs/git-config#Documentation/git-config.txt-httpversion" rel="nofollow noreferrer">git config http.version</a> (or just removing some larger files)</p>
  <hr />
  <h2><em><strong>UPDATE</strong></em></h2>
  <p>it does not actually seem to be about the self-hosted agent but a more general docker network cfg issue within my corporate network.</p>
  <p>Running the following works consistently fast on my VPN machine (running from home):</p>
  <pre><code>docker run -it
  ubuntu bash -c &quot;apt-get update; apt-get install -y wget; start=$SECONDS;
  wget http://cdimage.ubuntu.com/lubuntu/releases/18.04/release/lubuntu-18.04-alternate-amd64.iso;
  echo Duration: $(( SECONDS - start )) seconds&quot;
  </code></pre>
  <p>Comparision with powershell download (on the host):</p>
  <pre><code>$start=Get-Date
  $(New-Object
  net.webclient).Downloadfile(&quot;http://cdimage.ubuntu.com/lubuntu/releases/18.04/release/lubuntu-18.04-alternate-amd64.iso&quot;,
  &quot;e:/temp/lubuntu-18.04-alternate-amd64.iso&quot;)
  'Duration: {0:mm}
  min {0:ss} sec' -f ($(Get-Date)-$start)
  </code></pre>
  <p><em><strong>Corporate network</strong></em></p>
  <ul>
  <li>Docker: 1560 seconds (=26 min!)</li>
  <li>Windows host sys: Duration: 00 min 15 sec</li>
  </ul>
  <p><em><strong>Dev laptop (VPN, from home):</strong></em></p>
  <ul>
  <li>Docker: 144 seconds (=2min 24sec)</li>
  <li>Windows host sys: Duration: 02 min 16 sec</li>
  </ul>
  <p>Looking at the issues discussed in <a href="https://github.com/docker/for-win/issues/698" rel="nofollow noreferrer">https://github.com/docker/for-win/issues/698</a> (and proposed workaround which didn't work for me), it seems to be a non-trivial problem with Windows / hyper-v ..</p>