Answers: <p>I created some showcase for monorepo with <a href="https://gradle.org/" rel="nofollow noreferrer">Gradle</a> as build tool
  and <a href="https://circleci.com/" rel="nofollow noreferrer">CircleCI</a> or <a href="https://bitbucket.org/product/features/pipelines" rel="nofollow noreferrer">Bitbucket pipelines</a> as CI tool.</p>
  
  <p>It's based on the same ideas as in accepted answer and I used similar setup in two projects till now.</p>
  
  <p>See:
  <a href="https://github.com/zladovan/monorepo" rel="nofollow noreferrer">https://github.com/zladovan/monorepo</a></p>
  
  <p>Gradle specific stuff should be replaceable quite easily by something else. Just change <code>tools/ci/core/list-dependencies.sh</code>.</p>
  
Answers: CC BY-SA 4.0
Answers: 2020-04-04T16:01:08.137Z
Answers: Totally awesome!!!!  Added one issue to help so master branch never breaks to the project.
Answers: Dean Hiller
Answers: CC BY-SA 4.0
Answers: 2020-01-31T14:58:39.457Z
Answers: 2020-01-31T14:58:39.457Z
Answers: Ján Čabala
Answers: <p>The fact that all the code is in a single repository doesn't mean that all the code is changing every time a commit is pushed. I would first make the "pathways" in the code explicit. E.g. perhaps you have a few subdirectories:</p>
  
  <pre><code>- App1/
    - Docs/
    - Code/
    - Tests/
    - Makefile
  - App2/
    - Docs/
    - Code/
    - Tests/
    - Makefile
  </code></pre>
  
  <p>Assuming that App1 is independent from App2, you could trigger it's build by checking if anything in that directory changed (see <a href="https://stackoverflow.com/questions/424071/how-to-list-all-the-files-in-a-commit">https://stackoverflow.com/questions/424071/how-to-list-all-the-files-in-a-commit</a>).</p>
  
  <p>Most continuous integration tools have a git plugin or native feature that allows you to filter on a path. If you were using Jenkins, you could set up a few jobs, each with the included region to be be the pathway that should be triggered -- so, <em>e.g.</em> changes to <code>App1/*</code> would trigger <code>jobs/app1/</code> which executes <code>make -f App1/Makefile</code> or similar.</p>
  
  <p>The trick then to maintaining this project is really in the Makefiles, or other build tool of your choice. You would write the dependencies in there.</p>
  
  <p>You could also have a single global Makefile and use the <code>git diff-tree</code> trick in the question mentioned above:</p>
  
  <pre><code>if grep -q App1 `git diff-tree --no-commit-id --name-only -r &lt;commit&gt;` ; then 
    # trigger pipeline App1
    cd App1
    make build
    make test
    make publish
  fi
  </code></pre>
  
Answers: CC BY-SA 4.0
Answers: 2019-08-07T20:25:18.620Z
Answers: What if you change what's in the root directly like .gitlab-ci file? No build would be triggered because no file has been changed in App*
Answers: Kaymaz
Answers: CC BY-SA 4.0
Answers: 2019-08-07T20:27:46.373Z
Answers: of course the logic of the build trigger would have to accomodate _your_ use cases.
Answers: Bruce Becker
Answers: CC BY-SA 4.0
Answers: 2020-04-03T23:53:57.330Z
Answers: This is really missing the fact of libraries and if you build library A, you must also build services C and D that depend on A.  (I am trying to figure this out myself having worked at twitter which had it but they used pants build system to do it and pants kinda sucked...it was pretty slow)
Answers: Dean Hiller
Answers: CC BY-SA 4.0
Answers: 2019-08-07T06:53:44.987Z
Answers: 2019-08-07T06:53:44.987Z
Answers: Bruce Becker
Body: <p>Are there established CI/CD strategies &amp; best practices for working with a distributed team on a project that is utilizing monorepos?</p>
  
  <p>I can provide two specific use-cases:</p>
  
  <ol>
  <li><p>Working on content distribution, e.g. a daily/weekly HTML email newsletter, and</p></li>
  <li><p>A multi-project directory for building LaTeX documents</p></li>
  </ol>
  
  <p>Both of these scenarios will utilize the same CI/CD pipeline (build/send an email, compile and upload a LaTeX document) but with a growing &amp; variable amount of content that makes independent repos per document undesirable.</p>
  
  <p>Most of the projects using monorepos currently seem to be javascript projects with a lot of cross-dependencies on code. This is a step a way from that, but I'd still like to embrace the audit trail of version control with an automated development process for producing content.</p>
  
  <p>Do these projects compile every piece of code on change? Is there a way to embrace a git diff or use some sort of 'flag' file in the repo root to use CI/CD tools with a monorepo?</p>
  
Tags: <continuous-integration><git><travis-ci><gitlab-ci-runner>
Title: Git-based CI/CD Pipeline for Monorepo
UserDisplayName: Jared
ThreadText: # Git-based CI/CD Pipeline for Monorepo
  
  (Asked by: username_0 on 2019-07-27)
  
  Are there established CI/CD strategies & best practices for working with a distributed team on a project that is utilizing monorepos?
  
  I can provide two specific use-cases:
  
  1. Working on content distribution, e.g. a daily/weekly HTML email newsletter, and
  2. A multi-project directory for building LaTeX documents
  
  Both of these scenarios will utilize the same CI/CD pipeline (build/send an email, compile and upload a LaTeX document) but with a growing & variable amount of content that makes independent repos per document undesirable.
  
  Most of the projects using monorepos currently seem to be javascript projects with a lot of cross-dependencies on code. This is a step a way from that, but I'd still like to embrace the audit trail of version control with an automated development process for producing content.
  
  Do these projects compile every piece of code on change? Is there a way to embrace a git diff or use some sort of 'flag' file in the repo root to use CI/CD tools with a monorepo?
  
  **Tags:** continuous-integration, git, travis-ci, gitlab-ci-runner
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2019-08-07. Score: 3 | Accepted answer
  
  The fact that all the code is in a single repository doesn't mean that all the code is changing every time a commit is pushed. I would first make the "pathways" in the code explicit. E.g. perhaps you have a few subdirectories:
  
  ```
  - App1/
    - Docs/
    - Code/
    - Tests/
    - Makefile
  - App2/
    - Docs/
    - Code/
    - Tests/
    - Makefile
  
  ```
  
  Assuming that App1 is independent from App2, you could trigger it's build by checking if anything in that directory changed (see <https://stackoverflow.com/questions/424071/how-to-list-all-the-files-in-a-commit>).
  
  Most continuous integration tools have a git plugin or native feature that allows you to filter on a path. If you were using Jenkins, you could set up a few jobs, each with the included region to be be the pathway that should be triggered -- so, *e.g.* changes to `App1/*` would trigger `jobs/app1/` which executes `make -f App1/Makefile` or similar.
  
  The trick then to maintaining this project is really in the Makefiles, or other build tool of your choice. You would write the dependencies in there.
  
  You could also have a single global Makefile and use the `git diff-tree` trick in the question mentioned above:
  
  ```
  if grep -q App1 `git diff-tree --no-commit-id --name-only -r <commit>` ; then 
    # trigger pipeline App1
    cd App1
    make build
    make test
    make publish
  fi
  
  ```
  
  > **Comments:**
  > **username_4**: What if you change what's in the root directly like .gitlab-ci file? No build would be triggered because no file has been changed in App*  
  > **username_3**: of course the logic of the build trigger would have to accomodate _your_ use cases.  
  > **username_2**: This is really missing the fact of libraries and if you build library A, you must also build services C and D that depend on A.  (I am trying to figure this out myself having worked at twitter which had it but they used pants build system to do it and pants kinda sucked...it was pretty slow)  
  
  ---
  
  ### Answer by username_1 on 2020-01-31. Score: 1
  
  I created some showcase for monorepo with [Gradle](https://gradle.org/) as build tool
  and [CircleCI](https://circleci.com/) or [Bitbucket pipelines](https://bitbucket.org/product/features/pipelines) as CI tool.
  
  It's based on the same ideas as in accepted answer and I used similar setup in two projects till now.
  
  See:
  <https://github.com/zladovan/monorepo>
  
  Gradle specific stuff should be replaceable quite easily by something else. Just change `tools/ci/core/list-dependencies.sh`.
  
  > **Comments:**
  > **username_2**: Totally awesome!!!!  Added one issue to help so master branch never breaks to the project.  
  
  ---
  
  