Answers: <p><code>azcopy</code> (v10) tries to use all available memory to buffer the blobs on transfer. It seems that the calculated memory size is incorrect and exceeds the pod memory limit. </p>
  
  <p>This can be controlled with the option <code>AZCOPY_BUFFER_GB</code>, as described in the <code>azcopy env</code> output below. </p>
  
  <blockquote>
    <p>INFO: Name: AZCOPY_BUFFER_GB</p>
    
    <p>Description: Max number of GB that AzCopy should use for buffering data between network and disk. May include decimal point, e.g. 0.5.
    The default is based on machine size.</p>
  </blockquote>
  
  <p>So for a pod limit of 2GiB, you can set the buffer a bit lower: </p>
  
  <pre><code>export AZCOPY_BUFFER_GB=1.5
  azcopy bench ...
  </code></pre>
  
Answers: CC BY-SA 4.0
Answers: 2020-02-25T16:56:38.460Z
Answers: 2020-02-25T16:56:38.460Z
Answers: Synox
Body: <p>When running <code>azcopy copy</code> with huge repos, or when running <code>azcopy bench</code>, the pods are killed with the error <code>OOMKilled</code>.  (azcopy v10)</p>
  
  <p>The pod has to following resource limitations: </p>
  
  <pre><code>limits: { cpu: 1, memory: 2Gi }
  </code></pre>
  
Tags: <kubernetes><azure-devops><azure>
Title: How to avoid OOMKilled with azcopy on k8s
UserDisplayName: Synox
ThreadText: # How to avoid OOMKilled with azcopy on k8s
  
  (Asked by: username_0 on 2020-02-25)
  
  When running `azcopy copy` with huge repos, or when running `azcopy bench`, the pods are killed with the error `OOMKilled`. (azcopy v10)
  
  The pod has to following resource limitations:
  
  ```
  limits: { cpu: 1, memory: 2Gi }
  
  ```
  
  **Tags:** kubernetes, azure-devops, azure
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2020-02-25. Score: 1
  
  `azcopy` (v10) tries to use all available memory to buffer the blobs on transfer. It seems that the calculated memory size is incorrect and exceeds the pod memory limit.
  
  This can be controlled with the option `AZCOPY_BUFFER_GB`, as described in the `azcopy env` output below.
  
  > INFO: Name: AZCOPY\_BUFFER\_GB
  >
  > Description: Max number of GB that AzCopy should use for buffering data between network and disk. May include decimal point, e.g. 0.5.
  > The default is based on machine size.
  
  So for a pod limit of 2GiB, you can set the buffer a bit lower:
  
  ```
  export AZCOPY_BUFFER_GB=1.5
  azcopy bench ...
  
  ```
  
  
  ---
  
  