Answers: <p>I suspect that <a href="https://confluence.atlassian.com/bitbucket/build-test-and-deploy-with-pipelines-792496469.html" rel="nofollow noreferrer">Bitbucket pipelines</a> is enforcing this memory limit, not docker-compose.</p>
  <p>Similarly to docker-compose, the Bitbucket pipelines code is likely to use Linux CGroups to enforce the memory limit, through the Linux kernel's OOM killer.</p>
  <p>The page you linked is about memory limits in additional containers (e.g. for a database) used by the pipeline, not the pipeline itself.</p>
  <p>Some suggestions:</p>
  <ul>
  <li><p>Try editing <code>bitbucket-pipelines.yml</code> to include a <code>size</code> option for memory size - I would expect this is more useful than editing docker-compose options which may not be respected.  See <a href="https://confluence.atlassian.com/bitbucket/configure-bitbucket-pipelines-yml-792298910.html" rel="nofollow noreferrer">this doc page</a>.</p>
  </li>
  <li><p>Try running the build locally under Docker Compose to get an idea of memory usage.</p>
  </li>
  </ul>
  <p>It would help to include the details of your <code>bitbucket-pipelines.yml</code>, and the Bitbucket error message about being out of memory.</p>
  
Answers: CC BY-SA 4.0
Answers: 2019-01-01T17:34:23.677Z
Answers: 2023-02-23T12:49:12.540Z
Answers: 2023-02-23T12:49:12.540Z
Answers: RichVel
Answers: <p>When you limit resources in docker-compose, the docker engine prevent containers to use more than the limits, docker do this by Linux's Cgroups. 
  Its usage in docker-compose v3 is also straightforward: 
  <a href="https://docs.docker.com/compose/compose-file/#resources" rel="nofollow noreferrer">https://docs.docker.com/compose/compose-file/#resources</a></p>
  
  <p>When your OS is in OOM state, this kills processes automatically and maybe your containers are good candidates in the state, as the problem I have faced on my Gitlab CI's build machine to build node projects!</p>
  
Answers: CC BY-SA 4.0
Answers: 2018-12-29T12:04:44.420Z
Answers: 2018-12-29T12:04:44.420Z
Answers: AmirAhmad ZiaeePour
Body: <p>as one (sometimes failing) step of our pipeline we </p>
  
  <ul>
  <li>setup environment via installation of docker which runs 3 containers through own <code>docker-compose</code>. Unfortunately we could not figure out how to impose any memory limits on individual containers in docker-compose v3 config.</li>
  <li><p>run tests in <code>sbt</code> with  <code>SBT_OPTS="-Xmx1500m"</code> and every second to third time step is failing with OOM</p>
  
  <p>definitions:
    services:
      docker:
        memory: 1024</p></li>
  </ul>
  
  <p><a href="https://confluence.atlassian.com/bitbucket/use-services-and-databases-in-bitbucket-pipelines-874786688.html#UseservicesanddatabasesinBitbucketPipelines-Servicememorylimits" rel="nofollow noreferrer">Explanation</a> on the memory usage is not clear on this case.</p>
  
  <p>Could somebody please explain what's going on with the memory allocation to containers, docker host, and step env?</p>
  
  <p>Interesting is that running the same tests, same containers for multiple times usually gives different results. Understandable that real load on the physical server changes over time and running times can differ - but memory?</p>
  
  <p>Puzzled...</p>
  
Comments: CC BY-SA 4.0
Comments: 2019-01-31T21:24:53.303Z
Comments: Am I reading correctly that you are setting a memory limit of 1024 megabytes on the docker container and then running sbt in the container with a max memory size of 1500 megabytes inside of the container?
Comments: James Shewey
Comments: CC BY-SA 4.0
Comments: 2018-12-27T13:09:40.233Z
Comments: hi and welcome! does OOM stand here for "out of memory"?
Comments: Ta Mu
Comments: CC BY-SA 4.0
Comments: 2018-12-27T13:52:03.377Z
Comments: yes, precisely: https://en.wikipedia.org/wiki/Out_of_memory
Comments: xvga
Tags: <docker><docker-compose><bitbucket-pipelines>
Title: OOM when running Bitbucket pipelines
UserDisplayName: xvga
ThreadText: # OOM when running Bitbucket pipelines
  
  (Asked by: username_0 on 2018-12-27)
  
  as one (sometimes failing) step of our pipeline we
  
  * setup environment via installation of docker which runs 3 containers through own `docker-compose`. Unfortunately we could not figure out how to impose any memory limits on individual containers in docker-compose v3 config.
  * run tests in `sbt` with `SBT_OPTS="-Xmx1500m"` and every second to third time step is failing with OOM
  
    definitions:
    services:
    docker:
    memory: 1024
  
  [Explanation](https://confluence.atlassian.com/bitbucket/use-services-and-databases-in-bitbucket-pipelines-874786688.html#UseservicesanddatabasesinBitbucketPipelines-Servicememorylimits) on the memory usage is not clear on this case.
  
  Could somebody please explain what's going on with the memory allocation to containers, docker host, and step env?
  
  Interesting is that running the same tests, same containers for multiple times usually gives different results. Understandable that real load on the physical server changes over time and running times can differ - but memory?
  
  Puzzled...
  
  **Tags:** docker, docker-compose, bitbucket-pipelines
  
  > **Comments:**
  > **username_2**: hi and welcome! does OOM stand here for "out of memory"?  
  > **username_0**: yes, precisely: https://en.wikipedia.org/wiki/Out_of_memory  
  > **username_1**: Am I reading correctly that you are setting a memory limit of 1024 megabytes on the docker container and then running sbt in the container with a max memory size of 1500 megabytes inside of the container?  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2019-01-01. Score: 2
  
  I suspect that [Bitbucket pipelines](https://confluence.atlassian.com/bitbucket/build-test-and-deploy-with-pipelines-792496469.html) is enforcing this memory limit, not docker-compose.
  
  Similarly to docker-compose, the Bitbucket pipelines code is likely to use Linux CGroups to enforce the memory limit, through the Linux kernel's OOM killer.
  
  The page you linked is about memory limits in additional containers (e.g. for a database) used by the pipeline, not the pipeline itself.
  
  Some suggestions:
  
  * Try editing `bitbucket-pipelines.yml` to include a `size` option for memory size - I would expect this is more useful than editing docker-compose options which may not be respected. See [this doc page](https://confluence.atlassian.com/bitbucket/configure-bitbucket-pipelines-yml-792298910.html).
  * Try running the build locally under Docker Compose to get an idea of memory usage.
  
  It would help to include the details of your `bitbucket-pipelines.yml`, and the Bitbucket error message about being out of memory.
  
  
  ---
  
  ### Answer by username_4 on 2018-12-29. Score: 0
  
  When you limit resources in docker-compose, the docker engine prevent containers to use more than the limits, docker do this by Linux's Cgroups.
  Its usage in docker-compose v3 is also straightforward:
  <https://docs.docker.com/compose/compose-file/#resources>
  
  When your OS is in OOM state, this kills processes automatically and maybe your containers are good candidates in the state, as the problem I have faced on my Gitlab CI's build machine to build node projects!
  
  
  ---
  
  