Answers: <p>Hi and welcome to DevOps SE!</p>
  
  <p>While DevOps is not self-purposed, maybe it is worth to step back for a moment to find out the actual problem before trying out solution.</p>
  
  <p>As you have asked a more or less generic question, I'll give you sort of a "<em>DevOps primer</em>" stick more to methodology rather than give you a list of tools. </p>
  
  <p>How detailed you would tailor the implementation of this depends on many factors like business needs, complexity of teams, products and systems. .. and the time you have got/are allowed to think.</p>
  
  <ul>
  <li><strong>Why</strong> did they ask you to update DevOps techologies?</li>
  <li><strong>What</strong> is the business objective i.e. what will change after you are done - find out the management perspective. Will be software delivered faster/in shorter periods, or are deployment problems a real issue?</li>
  <li><p>If there is an update of DevOps technologies, what is the current <a href="https://en.wikipedia.org/wiki/DevOps_toolchain" rel="nofollow noreferrer">DevOps toolchain</a> (please read on this Wikipedia article to understand families of tools by their purpose in the toolchain) your colleagues use? </p></li>
  <li><p><strong>Who</strong> i.e. which team members have knowledge/daily usage of which of these tools?</p></li>
  <li><strong>What</strong> are technically software products your manufacture and deliver?</li>
  </ul>
  
  <p>Now you know the problems and the existing toolchain and products it produces:</p>
  
  <ul>
  <li>Are there now some quick fixes to achieve quick results or do you need more time to analyze data you have now, and find answers to questions you have to the team and management?</li>
  <li>Which tools do you think are missing and how which change would bring the company closer to the set objective?</li>
  <li>Will be free open source tools enough or some enterprise licenses could be required?</li>
  <li>Is it possible to design a DevOps strategy and generate iterative tasks to deliver changes and improve the situation in baby steps (<a href="https://en.wikipedia.org/wiki/Design_thinking" rel="nofollow noreferrer">design thinking</a> with your team as end users or do you have a complex situation in large enterprise - <a href="https://searchcio.techtarget.com/definition/systems-thinking" rel="nofollow noreferrer">system thinking</a>)?</li>
  </ul>
  
  <p>Side note: regarding to container management, depending on existing expertise you might consider using a solution bundling Kubernetes, e.g. OpenShift as already mentioned or RancherLabs etc. Regarding Jenkins, after you know what is happening, skip through its plugins, Jenkins-as-is is just a beginning.</p>
  
Answers: CC BY-SA 4.0
Answers: 2018-09-19T07:29:23.870Z
Answers: 2018-09-24T18:07:47.130Z
Answers: 2018-09-24T18:07:47.130Z
Answers: Ta Mu
Answers: <p>I've been responsible for setting up DevOps processes in two companies from scratch so hopefully this will help.</p>
  
  <p>Others have asked questions about business objectives but I'm going to give you a brief rundown of some of the problems we had and the technologies I used to fix it.</p>
  
  <ol>
  <li>Infrastructure Creation/Management was slow and error prone - I started creating and managing our infra (particularly K8's clusters) using <strong>Terraform</strong></li>
  <li>Kubernetes CI/CD becomes difficult as the number of services grows - to solve this issue we started using <strong>Helm</strong> which bills itself as the 'missing package manager' for Kubernetes, but it also helps manage Kubernetes .yaml files as templates which helps simplify CI/CD</li>
  <li>CI/CD - Building/testing and deploying applications was also slow and cumbersome, so we started using <strong>Jenkins (Blue Ocean)</strong> to define our deployment pipelines in a Jenkinsfile. Highly recommend this if you are still using the old version of Jenkins.</li>
  <li>Managing configurations of mutable infrastructure - we still have some 'mutable' infrastructure which was prone to drift, and you could use <strong>Chef</strong> or <strong>Ansible</strong> to manage those to minimise configuration drift or to bootstrap new servers quickly.</li>
  </ol>
  
Answers: CC BY-SA 4.0
Answers: 2019-01-17T22:07:13.950Z
Answers: Helm is good but Helmfile is awesome. Put a list of releases into helmfile.yaml into git and setup a jenkins job to trigger that just runs helmfile cli that will run helm for each release. Helmfile apply command will diff all the stuff in k8s against what is in git and only update what has changed. It supports encryption of values in yaml so the it's okay to have secrets in git.
Answers: simbo1905
Answers: CC BY-SA 4.0
Answers: 2018-09-19T17:02:52.927Z
Answers: what about binary repository managers to store artefacts coming from the CI, i.e. Artifactory or Nexus?
Answers: Ta Mu
Answers: CC BY-SA 4.0
Answers: 2018-09-19T17:29:13.853Z
Answers: Thanks. This was very useful. We already are using most, except the `Helm` one! Maybe I should introduce it to them!
Answers: Tina J
Answers: CC BY-SA 4.0
Answers: 2018-09-21T08:39:04.377Z
Answers: Peter I generally use Amazon S3 for artifact storage.
Answers: Caledonia91
Answers: CC BY-SA 4.0
Answers: 2018-09-19T16:53:17.073Z
Answers: 2018-09-19T16:53:17.073Z
Answers: Caledonia91
Answers: <p>If you looking for a fun idea try adding a chatbot to drive your devops automation:</p>
  
  <p><a href="https://i.sstatic.net/bXKC4.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/bXKC4.jpg" alt="slackbot deploy"></a></p>
  
  <p><a href="https://i.sstatic.net/O3MFx.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/O3MFx.jpg" alt="slackbot regex"></a></p>
  
  <p><a href="https://i.sstatic.net/mhn9C.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/mhn9C.jpg" alt="Slackbot architecture"></a></p>
  
  <p>I wrote a <a href="https://www.slideshare.net/simbo1905/chatbots-for-selfservice-devops?from_m_app=ios" rel="nofollow noreferrer">short presentation</a> with a link at the end to a video of settings up a chatbot on slack but botkit supports a lot of different chat solutions. </p>
  
Answers: CC BY-SA 4.0
Answers: 2019-01-17T22:20:11.187Z
Answers: 2019-01-17T22:30:55.213Z
Answers: 2019-01-17T22:30:55.213Z
Answers: simbo1905
Answers: <p>Alright... you're not going to implement every tool or automation at the same time, as some of them have quite deep impact on your development processes (and I daresay, development culture). Take a step-by-step approach; research each individual tool. Figure out what it actually does, what it is useful for. Play around with them, install them locally, go through tutorials, and see what they do, or whip up a quick VM somewhere (AWS, etc.) if you need more than one.</p>
  
  <p>Then, take a look at the work your team/company is doing right now. What <em>are</em> the manual process you wish to automate? Ideas should spring up pretty quickly.</p>
  
  <p>A nice, simple(-ish), bog-standard stack to start out with:</p>
  
  <ul>
  <li>Vagrant to automate setting up VMs for your developers, or test environments.</li>
  <li>Ansible to automate configuration management (relatively lightweight compared with Chef or Puppet)</li>
  <li>Gitlab or Jenkins as your automated CI/CD driver, mostly depending on taste</li>
  <li>Docker for your containers</li>
  <li>docker-compose, docker-swarm, Kubernetes, OpenShift to automate container deployment</li>
  <li>ELK or Splunk for your logging/monitoring, again depending on taste</li>
  </ul>
  
  <p>Have fun!</p>
  
Answers: CC BY-SA 4.0
Answers: 2018-09-19T07:10:32.027Z
Answers: 2018-09-19T07:10:32.027Z
Answers: AnoE
Body: <p>This is a general question, but couldn't find a better place to ask. I'm a new DevOps member in a company, and I've been ask to "update" our DevOps technologies. </p>
  
  <p>Generally speaking, in what areas do we need to provide tools, and what are some latest and advanced technologies/tools to do that?</p>
  
  <p>For instance, we need container management (e.g. <code>docker, kubernetes</code>), log analysis (e.g. <code>ELK Stack</code>), Application monitoring (e.g. <code>JMX</code>), <code>Weblogic</code>, database management, <code>Jenkins</code>, etc.? </p>
  
  <p>What else, and what tools? Which processes are probably needed to be automated? It can be anything. I just need some advice. </p>
  
Comments: CC BY-SA 4.0
Comments: 2018-09-18T22:42:56.187Z
Comments: Hi Tina, this is indeed quite generic question. I would suggest two things. First, to split this question into specific areas describing parts of your stack and ask for best practices for automation in that specific area, based on what you have. Second, I think you might need to suggest to your manager that you need someone experienced to consult your current state of architecture with, get them to pay someone (internal or external) for a week of advice or so, then you'd have some specific points that you could work on for the next few months.
Comments: Jiri Klouda
Comments: CC BY-SA 4.0
Comments: 2018-09-18T22:45:25.847Z
Comments: One more suggestion, articles on internet on best practices are often marketing content written by specific vendors to push their product. Unless an article discusses multiple solutions by multiple vendors with unclear winner and case based suggestions where in different case different  vendor is a recommendation, then I think you might be quite sure you are reading a marketing document.
Comments: Jiri Klouda
Comments: CC BY-SA 4.0
Comments: 2018-09-18T22:47:00.747Z
Comments: I get the splitting the question part. But I hope I can get some best practices and names here.
Comments: Tina J
Comments: CC BY-SA 4.0
Comments: 2018-09-18T22:55:21.527Z
Comments: You might, but this is simply something that is best discussed in a specific case on a whiteboard. Often the things to do depend also on what resources are available to your team, what skills you have, what kind of advice you can get in the company, what other people know or are familiar with. It does not matter if one tool is slightly better than the other if you'd struggle with one and have easier work with the other based for example on the programming language the tool uses for modules.
Comments: Jiri Klouda
Tags: <amazon-web-services><deployment><containers><automation><infrastructure-as-code>
Title: DevOps tools and automation of manual processes
UserDisplayName: Tina J
ThreadText: # DevOps tools and automation of manual processes
  
  (Asked by: username_0 on 2018-09-18)
  
  This is a general question, but couldn't find a better place to ask. I'm a new DevOps member in a company, and I've been ask to "update" our DevOps technologies.
  
  Generally speaking, in what areas do we need to provide tools, and what are some latest and advanced technologies/tools to do that?
  
  For instance, we need container management (e.g. `docker, kubernetes`), log analysis (e.g. `ELK Stack`), Application monitoring (e.g. `JMX`), `Weblogic`, database management, `Jenkins`, etc.?
  
  What else, and what tools? Which processes are probably needed to be automated? It can be anything. I just need some advice.
  
  **Tags:** amazon-web-services, deployment, containers, automation, infrastructure-as-code
  
  > **Comments:**
  > **username_1**: Hi username_0, this is indeed quite generic question. I would suggest two things. First, to split this question into specific areas describing parts of your stack and ask for best practices for automation in that specific area, based on what you have. Second, I think you might need to suggest to your manager that you need someone experienced to consult your current state of architecture with, get them to pay someone (internal or external) for a week of advice or so, then you'd have some specific points that you could work on for the next few months.  
  > **username_1**: One more suggestion, articles on internet on best practices are often marketing content written by specific vendors to push their product. Unless an article discusses multiple solutions by multiple vendors with unclear winner and case based suggestions where in different case different  vendor is a recommendation, then I think you might be quite sure you are reading a marketing document.  
  > **username_0**: I get the splitting the question part. But I hope I can get some best practices and names here.  
  > **username_1**: You might, but this is simply something that is best discussed in a specific case on a whiteboard. Often the things to do depend also on what resources are available to your team, what skills you have, what kind of advice you can get in the company, what other people know or are familiar with. It does not matter if one tool is slightly better than the other if you'd struggle with one and have easier work with the other based for example on the programming language the tool uses for modules.  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_5 on 2018-09-19. Score: 5 | Accepted answer
  
  Alright... you're not going to implement every tool or automation at the same time, as some of them have quite deep impact on your development processes (and I daresay, development culture). Take a step-by-step approach; research each individual tool. Figure out what it actually does, what it is useful for. Play around with them, install them locally, go through tutorials, and see what they do, or whip up a quick VM somewhere (AWS, etc.) if you need more than one.
  
  Then, take a look at the work your team/company is doing right now. What *are* the manual process you wish to automate? Ideas should spring up pretty quickly.
  
  A nice, simple(-ish), bog-standard stack to start out with:
  
  * Vagrant to automate setting up VMs for your developers, or test environments.
  * Ansible to automate configuration management (relatively lightweight compared with Chef or Puppet)
  * Gitlab or Jenkins as your automated CI/CD driver, mostly depending on taste
  * Docker for your containers
  * docker-compose, docker-swarm, Kubernetes, OpenShift to automate container deployment
  * ELK or Splunk for your logging/monitoring, again depending on taste
  
  Have fun!
  
  
  ---
  
  ### Answer by username_2 on 2018-09-19. Score: 4
  
  Hi and welcome to DevOps SE!
  
  While DevOps is not self-purposed, maybe it is worth to step back for a moment to find out the actual problem before trying out solution.
  
  As you have asked a more or less generic question, I'll give you sort of a "*DevOps primer*" stick more to methodology rather than give you a list of tools.
  
  How detailed you would tailor the implementation of this depends on many factors like business needs, complexity of teams, products and systems. .. and the time you have got/are allowed to think.
  
  * **Why** did they ask you to update DevOps techologies?
  * **What** is the business objective i.e. what will change after you are done - find out the management perspective. Will be software delivered faster/in shorter periods, or are deployment problems a real issue?
  * If there is an update of DevOps technologies, what is the current [DevOps toolchain](https://en.wikipedia.org/wiki/DevOps_toolchain) (please read on this Wikipedia article to understand families of tools by their purpose in the toolchain) your colleagues use?
  * **Who** i.e. which team members have knowledge/daily usage of which of these tools?
  * **What** are technically software products your manufacture and deliver?
  
  Now you know the problems and the existing toolchain and products it produces:
  
  * Are there now some quick fixes to achieve quick results or do you need more time to analyze data you have now, and find answers to questions you have to the team and management?
  * Which tools do you think are missing and how which change would bring the company closer to the set objective?
  * Will be free open source tools enough or some enterprise licenses could be required?
  * Is it possible to design a DevOps strategy and generate iterative tasks to deliver changes and improve the situation in baby steps ([design thinking](https://en.wikipedia.org/wiki/Design_thinking) with your team as end users or do you have a complex situation in large enterprise - [system thinking](https://searchcio.techtarget.com/definition/systems-thinking))?
  
  Side note: regarding to container management, depending on existing expertise you might consider using a solution bundling Kubernetes, e.g. OpenShift as already mentioned or RancherLabs etc. Regarding Jenkins, after you know what is happening, skip through its plugins, Jenkins-as-is is just a beginning.
  
  
  ---
  
  ### Answer by username_3 on 2018-09-19. Score: 3
  
  I've been responsible for setting up DevOps processes in two companies from scratch so hopefully this will help.
  
  Others have asked questions about business objectives but I'm going to give you a brief rundown of some of the problems we had and the technologies I used to fix it.
  
  1. Infrastructure Creation/Management was slow and error prone - I started creating and managing our infra (particularly K8's clusters) using **Terraform**
  2. Kubernetes CI/CD becomes difficult as the number of services grows - to solve this issue we started using **Helm** which bills itself as the 'missing package manager' for Kubernetes, but it also helps manage Kubernetes .yaml files as templates which helps simplify CI/CD
  3. CI/CD - Building/testing and deploying applications was also slow and cumbersome, so we started using **Jenkins (Blue Ocean)** to define our deployment pipelines in a Jenkinsfile. Highly recommend this if you are still using the old version of Jenkins.
  4. Managing configurations of mutable infrastructure - we still have some 'mutable' infrastructure which was prone to drift, and you could use **Chef** or **Ansible** to manage those to minimise configuration drift or to bootstrap new servers quickly.
  
  > **Comments:**
  > **username_2**: what about binary repository managers to store artefacts coming from the CI, i.e. Artifactory or Nexus?  
  > **username_0**: Thanks. This was very useful. We already are using most, except the `Helm` one! Maybe I should introduce it to them!  
  > **username_3**: Peter I generally use Amazon S3 for artifact storage.  
  > **username_4**: Helm is good but Helmfile is awesome. Put a list of releases into helmfile.yaml into git and setup a jenkins job to trigger that just runs helmfile cli that will run helm for each release. Helmfile apply command will diff all the stuff in k8s against what is in git and only update what has changed. It supports encryption of values in yaml so the it's okay to have secrets in git.  
  
  ---
  
  ### Answer by username_4 on 2019-01-17. Score: 0
  
  If you looking for a fun idea try adding a chatbot to drive your devops automation:
  
  [![slackbot deploy](https://i.sstatic.net/bXKC4.jpg)](https://i.sstatic.net/bXKC4.jpg)
  
  [![slackbot regex](https://i.sstatic.net/O3MFx.jpg)](https://i.sstatic.net/O3MFx.jpg)
  
  [![Slackbot architecture](https://i.sstatic.net/mhn9C.jpg)](https://i.sstatic.net/mhn9C.jpg)
  
  I wrote a [short presentation](https://www.slideshare.net/username_4/chatbots-for-selfservice-devops?from_m_app=ios) with a link at the end to a video of settings up a chatbot on slack but botkit supports a lot of different chat solutions.
  
  
  ---
  
  