Answers: <p>I can answer bout docker, one of the best practice to use docker is to keep the docker file and the compose files in the same repository of the project, so wherever you clone the project you can build the docker image, and it is good to keep multiple versions of docker compose files for example (prod, staging, dev) so you can build the image and run the container with specific option for each env for example for dev machine you can use specific network and run more dependencies container or whatever.</p>
  
Answers: CC BY-SA 3.0
Answers: 2017-08-24T03:30:38.267Z
Answers: 2017-08-24T03:30:38.267Z
Answers: Wissam Roujoulah
Answers: <p>Each tool's code goes into its own repo. for e.g</p>
  
  <ol>
  <li>Jenkins Groovy template into a Jenkins repo</li>
  <li>Ansible YAML playbooks in its own repo (with the roles , tasks , inventory sub directories</li>
  <li>Cloudformation / Terrform templates in its own repo</li>
  <li>Docker files in its own
  5.. And So on</li>
  </ol>
  
  <p>This would help you scale better in terms of process orchestration and maintaining various branches for each environment</p>
  
  <p>This would give you more granular control and offload all your versioning overhead to the version control systems. Also create separate branches for each environment and tag the code for every production release (as we do for application code base). Think Infra and process in terms of code. (Any change in process has to be codified and sent to QA, SIT , UAT and then to PROD) similar to application.</p>
  
  <p>For e.g you may have V2.1 of Ansible running in Production ( master branch) but V2.0 of docker containers running in Prod (master branch) </p>
  
  <p>Similarly keep your DB scripts / bash scripts in their own repositories and maybe you can have a healthcheck file (JSON/YAML) configured to show the versions of all the tools / parts in each deployed URL for tracking and automating purposes. (So that your webhooks reads the URL and automates deployments)</p>
  
Answers: CC BY-SA 3.0
Answers: 2017-08-25T05:02:18.487Z
Answers: The pitfall of this approach is, v2.1 is in qa and not validated and you have to patch production urgently, you can't modify v2.0 and if you make a v2.2 for this patch there's a high risk of it being lost or overwritten when v2.1 goes to production, multiply by the amount of separate code in a repo and you soon have a nightmare of backports (it works, but I had to add this disclaimer :))
Answers: Tensibai
Answers: CC BY-SA 3.0
Answers: 2017-08-29T13:03:48.720Z
Answers: Using branches to keep track of environment/deployment-specific information looks like an ant-pattern to me: if we have 20 environments, this  means that we have 20 branches to keep in sync… a likely source of errors and confusion. Could you explain why you do not use configuration files to keep track of environment/deployment-specific information and what is your workflow working with these branches? These are not trivial problems!
Answers: Michaël Le Barbier
Answers: CC BY-SA 3.0
Answers: 2017-08-24T19:19:57.703Z
Answers: 2017-08-24T19:19:57.703Z
Answers: Ameen Ibrahim Raffic - 'AIR'
Answers: <p>Making a distinction between Ops, Dev and DevOps promotes isolation and enforces a "throw it over the wall" mindset. To increase cooperation between teams one should put everything in a repository that is required to build and deploy the project. </p>
  
  <p>Having said that, the answer to the question: </p>
  
  <blockquote>
    <p>How to structure DevOps related code and configs in a code repository?</p>
  </blockquote>
  
  <p>is that if config is required to run the project then one should put it in the same directory.</p>
  
Answers: CC BY-SA 3.0
Answers: 2017-08-25T21:54:50.907Z
Answers: 2017-08-25T22:02:11.067Z
Answers: 2017-08-25T22:02:11.067Z
Answers: 030
Answers: <p>The current organisation of the code and configuration you describe is structured by the technical solutions involved.  This is a bad design that will add a lot of overhead in our maintenance activities and will add a lot of traps in our way as well. Instead, that organisation should be structured around the <a href="https://devops.stackexchange.com/questions/466/what-is-an-artifact-or-artefact">artefacts</a> we are deploying.</p>
  
  <p>The reason for this is that we want to consider artefacts (<em>e.g.</em> a docker image or a software package) as the objects of the following verbs:</p>
  
  <ul>
  <li>build</li>
  <li>test</li>
  <li>deploy</li>
  </ul>
  
  <p>to consider a minimal set of automated tasks we want to perform. If we want to change something about how the test verb is implemented, it is easy to visit the folder corresponding to that artefact in the appropriate repository and then discover the jenkins-specific automation items that needs to be updated.  Instead, if the automation recipes are structured around technical solutions, then we need to figure out of the blue that jenkins is involved in the test procedures and find there the artefact related automation items.  In complex situations, the organisation around technical solutions makes updates very hard, because we have to know a priori all the technical solutions involved in some service to update them accordingly.</p>
  
  <p>For instance a repository containing the code for a website and a micro-service “a” could have the following sub-directories dedicated to operations:</p>
  
  <pre><code>./ops/website
  ./ops/micro-service-a
  </code></pre>
  
  <p>each having three scripts called <code>build</code>, <code>test</code> and <code>deploy</code>.  Now that the organisation of automation items has somehow been clarified, let's turn our attention to configuration.</p>
  
  <p>The main conditions and requirements about the configuration organisation are set by the <code>deploy</code> verb when applied on a service-like artefact.  The <code>deploy</code> verb should have the following parameters:</p>
  
  <ul>
  <li>the version of the artefact to deploy,</li>
  <li>the deployment target of the artefact, which describes the concrete environment where the deployed artefact will run (<em>e.g.</em> a cluster and endpoints it should talk to)</li>
  <li>the credentials it should use to connect to other endpoints (<em>e.g.</em> databases)</li>
  <li>the runtime configuration of (like how long cache entries should live, etc.)</li>
  </ul>
  
  <p>From the operational perspective, this breakdown of the parametrisation matches the natural degrees of freedom of the deployment problem – aside from the credentials that could be bundled with the runtime configuration, but it is better to separate them to avoid spreading them carelessly.</p>
  
Answers: CC BY-SA 3.0
Answers: 2017-08-29T12:55:52.377Z
Answers: 2017-08-29T13:06:48.840Z
Answers: 2017-08-29T13:06:48.840Z
Answers: Michaël Le Barbier
Body: <p>We've been growing as a company, our products are expanding and our DevOps related activities and efforts are growing as well - we've switched from Bamboo to a more flexible and configurable Jenkins, using deployment pipelines and other plugins; switched to Ansible and starting to use Docker here and there internally.</p>
  
  <p>All of these things require some level of coding or configuration - Ansible scripts and configs, Jenkins groovy scripts, Dockerfiles and YAML configs. </p>
  
  <p>For now, we've created a separate "ops" repository with high-level directories for <code>jenkins</code>, <code>ansible</code>, <code>docker</code> and <code>other</code> (which is a terrible name, but for now all "other" DevOps automation things are there).</p>
  
  <p>Our approach does not feel right and might not scale, but what are the best practices and recommendations to keep DevOps-related code in a code repository or repositories?</p>
  
Comments: CC BY-SA 3.0
Comments: 2017-08-24T08:24:00.563Z
Comments: I go with the "each part is an app, one repo per app" method, in chef that means 1 repo per cookbook.
Comments: Tensibai
Comments: CC BY-SA 3.0
Comments: 2017-08-24T16:15:51.800Z
Comments: @Tensibai right, I was afraid that a single "ops" repo would quickly become impractical. Thanks.
Comments: alecxe
Comments: CC BY-SA 3.0
Comments: 2017-08-24T19:11:09.873Z
Comments: That has been the legacy form of cookbook management in chef, 1 repo with all cookbook and proven a footgun in most cases hence the change, but I'm less comfortable with ansible to say it would fit also, Jenkins pipelines (if v2) and docker files should live with the project they handle IMO, and I've no idea of what you put under other so I can't really give any advice here
Comments: Tensibai
Comments: CC BY-SA 3.0
Comments: 2017-08-24T20:07:46.590Z
Comments: @Tensibai got it! Other consists of mostly bash and python utilities or periodically executed scripts for multiple internal tools..they don't really fit anywhere and we could not think of a better place than "other"..I'll see if I can post the contents of the directories into the question as well. Thanks.
Comments: alecxe
Comments: CC BY-SA 3.0
Comments: 2017-08-29T13:32:42.103Z
Comments: I'd split them in multiple repo by 'affinity' of work, scripts working on app X together, you may have a script used on two apps, but if app A change in a way the script has to handle which app it talks to, it's better to have two separated versions, so ATEOTD I'd store them with the application they relate to or if they span multiples application in a specific repository per task, so you always have a version in line with the deployed applications and you don't need to tag an unrelated script at the same time.
Comments: Tensibai
Tags: <jenkins><docker><ansible><git><repository>
Title: How to structure DevOps related code and configs in a code repository?
UserDisplayName: alecxe
ThreadText: # How to structure DevOps related code and configs in a code repository?
  
  (Asked by: username_0 on 2017-08-24)
  
  We've been growing as a company, our products are expanding and our DevOps related activities and efforts are growing as well - we've switched from Bamboo to a more flexible and configurable Jenkins, using deployment pipelines and other plugins; switched to Ansible and starting to use Docker here and there internally.
  
  All of these things require some level of coding or configuration - Ansible scripts and configs, Jenkins groovy scripts, Dockerfiles and YAML configs.
  
  For now, we've created a separate "ops" repository with high-level directories for `jenkins`, `ansible`, `docker` and `other` (which is a terrible name, but for now all "other" DevOps automation things are there).
  
  Our approach does not feel right and might not scale, but what are the best practices and recommendations to keep DevOps-related code in a code repository or repositories?
  
  **Tags:** jenkins, docker, ansible, git, repository
  
  > **Comments:**
  > **username_1**: I go with the "each part is an app, one repo per app" method, in chef that means 1 repo per cookbook.  
  > **username_0**: @username_1 right, I was afraid that a single "ops" repo would quickly become impractical. Thanks.  
  > **username_1**: That has been the legacy form of cookbook management in chef, 1 repo with all cookbook and proven a footgun in most cases hence the change, but I'm less comfortable with ansible to say it would fit also, Jenkins pipelines (if v2) and docker files should live with the project they handle IMO, and I've no idea of what you put under other so I can't really give any advice here  
  > **username_0**: @username_1 got it! Other consists of mostly bash and python utilities or periodically executed scripts for multiple internal tools..they don't really fit anywhere and we could not think of a better place than "other"..I'll see if I can post the contents of the directories into the question as well. Thanks.  
  > **username_1**: I'd split them in multiple repo by 'affinity' of work, scripts working on app X together, you may have a script used on two apps, but if app A change in a way the script has to handle which app it talks to, it's better to have two separated versions, so ATEOTD I'd store them with the application they relate to or if they span multiples application in a specific repository per task, so you always have a version in line with the deployed applications and you don't need to tag an unrelated script at the same time.  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_4 on 2017-08-29. Score: 4 | Accepted answer
  
  The current organisation of the code and configuration you describe is structured by the technical solutions involved. This is a bad design that will add a lot of overhead in our maintenance activities and will add a lot of traps in our way as well. Instead, that organisation should be structured around the [artefacts](https://devops.stackexchange.com/questions/466/what-is-an-artifact-or-artefact) we are deploying.
  
  The reason for this is that we want to consider artefacts (*e.g.* a docker image or a software package) as the objects of the following verbs:
  
  * build
  * test
  * deploy
  
  to consider a minimal set of automated tasks we want to perform. If we want to change something about how the test verb is implemented, it is easy to visit the folder corresponding to that artefact in the appropriate repository and then discover the jenkins-specific automation items that needs to be updated. Instead, if the automation recipes are structured around technical solutions, then we need to figure out of the blue that jenkins is involved in the test procedures and find there the artefact related automation items. In complex situations, the organisation around technical solutions makes updates very hard, because we have to know a priori all the technical solutions involved in some service to update them accordingly.
  
  For instance a repository containing the code for a website and a micro-service “a” could have the following sub-directories dedicated to operations:
  
  ```
  ./ops/website
  ./ops/micro-service-a
  
  ```
  
  each having three scripts called `build`, `test` and `deploy`. Now that the organisation of automation items has somehow been clarified, let's turn our attention to configuration.
  
  The main conditions and requirements about the configuration organisation are set by the `deploy` verb when applied on a service-like artefact. The `deploy` verb should have the following parameters:
  
  * the version of the artefact to deploy,
  * the deployment target of the artefact, which describes the concrete environment where the deployed artefact will run (*e.g.* a cluster and endpoints it should talk to)
  * the credentials it should use to connect to other endpoints (*e.g.* databases)
  * the runtime configuration of (like how long cache entries should live, etc.)
  
  From the operational perspective, this breakdown of the parametrisation matches the natural degrees of freedom of the deployment problem – aside from the credentials that could be bundled with the runtime configuration, but it is better to separate them to avoid spreading them carelessly.
  
  
  ---
  
  ### Answer by username_2 on 2017-08-24. Score: 6
  
  I can answer bout docker, one of the best practice to use docker is to keep the docker file and the compose files in the same repository of the project, so wherever you clone the project you can build the docker image, and it is good to keep multiple versions of docker compose files for example (prod, staging, dev) so you can build the image and run the container with specific option for each env for example for dev machine you can use specific network and run more dependencies container or whatever.
  
  
  ---
  
  ### Answer by username_3 on 2017-08-24. Score: 4
  
  Each tool's code goes into its own repo. for e.g
  
  1. Jenkins Groovy template into a Jenkins repo
  2. Ansible YAML playbooks in its own repo (with the roles , tasks , inventory sub directories
  3. Cloudformation / Terrform templates in its own repo
  4. Docker files in its own
     5.. And So on
  
  This would help you scale better in terms of process orchestration and maintaining various branches for each environment
  
  This would give you more granular control and offload all your versioning overhead to the version control systems. Also create separate branches for each environment and tag the code for every production release (as we do for application code base). Think Infra and process in terms of code. (Any change in process has to be codified and sent to QA, SIT , UAT and then to PROD) similar to application.
  
  For e.g you may have V2.1 of Ansible running in Production ( master branch) but V2.0 of docker containers running in Prod (master branch)
  
  Similarly keep your DB scripts / bash scripts in their own repositories and maybe you can have a healthcheck file (JSON/YAML) configured to show the versions of all the tools / parts in each deployed URL for tracking and automating purposes. (So that your webhooks reads the URL and automates deployments)
  
  > **Comments:**
  > **username_1**: The pitfall of this approach is, v2.1 is in qa and not validated and you have to patch production urgently, you can't modify v2.0 and if you make a v2.2 for this patch there's a high risk of it being lost or overwritten when v2.1 goes to production, multiply by the amount of separate code in a repo and you soon have a nightmare of backports (it works, but I had to add this disclaimer :))  
  > **username_4**: Using branches to keep track of environment/deployment-specific information looks like an ant-pattern to me: if we have 20 environments, this  means that we have 20 branches to keep in sync… a likely source of errors and confusion. Could you explain why you do not use configuration files to keep track of environment/deployment-specific information and what is your workflow working with these branches? These are not trivial problems!  
  
  ---
  
  ### Answer by username_5 on 2017-08-25. Score: 3
  
  Making a distinction between Ops, Dev and DevOps promotes isolation and enforces a "throw it over the wall" mindset. To increase cooperation between teams one should put everything in a repository that is required to build and deploy the project.
  
  Having said that, the answer to the question:
  
  > How to structure DevOps related code and configs in a code repository?
  
  is that if config is required to run the project then one should put it in the same directory.
  
  
  ---
  
  