Answers: <p>Test Kitchen has a kitchen-ansible provisioner plugin for testing of Ansible code. It isn't as deep as the Chef integration but it does get the job done for most cases. There is also the more recent Molecule project which is a dedicated Ansible testing system.</p>
  
Answers: CC BY-SA 3.0
Answers: 2017-03-01T00:53:39.797Z
Answers: 2017-03-01T00:53:39.797Z
Answers: coderanger
Answers: <p>The two tools I've seen for this are <a href="https://www.chef.io/inspec/" rel="nofollow noreferrer">InSpec</a> and <a href="http://serverspec.org/" rel="nofollow noreferrer">ServerSpec</a>.  Serverspec is a Ruby-based tool that builds on <a href="http://rspec.info/" rel="nofollow noreferrer">RSpec</a>.  InSpec is inspired by RSpec and ServerSpec.</p>
  
  <p>I've used ServerSpec. It's cool, but maybe not 100% stable. I've had problems with testing for specific versions of software on Ubuntu.</p>
  
  <p>I've read the InSpec docs but haven't dug in deep.  It does essentially the same thing as Serverspec.  </p>
  
  <p>Judging by the Github commits, it looks like work on ServerSpec has tailed off somewhat, whereas InSpec is just getting ramped up.</p>
  
  <p><strong>UPDATE:</strong> InSpec 4.x or above uses a mixed commercial / open source license - see comments.</p>
  
Answers: CC BY-SA 3.0
Answers: 2017-03-08T01:10:03.563Z
Answers: well "not 100% stable" for a testing tool does not sound any good
Answers: ᴳᵁᴵᴰᴼ
Answers: CC BY-SA 4.0
Answers: 2018-05-06T05:54:12.900Z
Answers: InSpec is very good as a test tool, and makes it easy to install nothing on the server, whereas ServerSpec can only do this with some extra work.   Some work would be needed to have InSpec use an Ansible inventory though - probably easier if inventory is in [YAML format](http://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html) (Ansible 2.4+).
Answers: RichVel
Answers: CC BY-SA 3.0
Answers: 2017-03-01T00:49:48.590Z
Answers: To slightly clarify, InSpec emulates RSpec but does not build on it.
Answers: coderanger
Answers: CC BY-SA 3.0
Answers: 2017-03-01T03:16:13.837Z
Answers: @MattSchuchard Thanks for that reference!
Answers: Dave Swersky
Answers: CC BY-SA 4.0
Answers: 2019-10-16T10:57:36.093Z
Answers: InSpec update - current versions (4.x and higher) are now **commercial** requiring payment, though underlying source code remains open source.  If using for personal use, non-commercial use, or 'experimental' in a business, InSpec is still free, but in other cases they require a commercial license.  See [Chef licensing docs page](https://docs.chef.io/chef_license.html) and EULA.  Beware the confusing "Chef is 100% open source" statements from Apr 2019 - technically true but executable tools using Chef trademarks are subject to EULA.
Answers: RichVel
Answers: CC BY-SA 4.0
Answers: 2017-02-28T20:44:26.593Z
Answers: 2019-10-17T14:18:59.083Z
Answers: 2019-10-17T14:18:59.083Z
Answers: Dave Swersky
Answers: <p>When using configuration management tools, such as Ansible, the tool itself would be responsible preventing configuration drift. Once you used Ansible to set a certain configuration, repetitive execution of Ansible will ensure your configuration is as you defined it to be. This also requires your Ansible code to be written in a manner which is idempotent.</p>
  
  <p>Given the above, testing provisioning can be achieved by executing your Ansible playbooks in a loop from some server. For example, a cron job, or Jenkins, can execute the playbooks every 30 minutes and report back on any failure. Having no failures means that your configuration is in check, having failures means that there is a problem getting servers into your <em>desired state</em>.</p>
  
  <p>In a case where you cannot trust your code to be written as idempotent, and thus you cannot really execute Ansible repetitively in a loop from an automatic server a workaround exists. You can do the same as above (run Ansible in a loop) but use its <a href="http://docs.ansible.com/ansible/playbooks_checkmode.html" rel="noreferrer"><em>dry run</em> mode</a>. Each time Ansible reports that changes are required, the Jenkins job (or cron job) can notify you that your provisioned configuration has been altered and servers are not in their <em>desired state</em>.</p>
  
  <p>To make sure that your Ansible code is actually doing what you think it should be doing, the solutions mentioned by Dave Swersky apply. Both <a href="https://www.chef.io/inspec/" rel="noreferrer">InSpec</a> and <a href="http://serverspec.org/" rel="noreferrer">Serverspec</a> are tools that verify in a <em>different</em> way that your playbooks actually does what you mean. A great way to execute these kinds of tools in test environments (even docker containers) is to use <a href="http://kitchen.ci/" rel="noreferrer">kitchen.ci</a> which handles all the glue between the various infra unit testing tools, and execution of your playbooks/modules/cookbooks.</p>
  
  <p>Kitchen.ci was originally used to test Chef cookbooks, but <a href="https://github.com/neillturner/kitchen-ansible" rel="noreferrer">plugins exist for Ansible</a> and other CM tools as well.</p>
  
Answers: CC BY-SA 3.0
Answers: 2017-03-01T03:32:20.357Z
Answers: 2017-08-24T11:47:24.157Z
Answers: 2017-08-24T11:47:24.157Z
Answers: Evgeny Zislis
Answers: <p>It is usual to measure the code coverage for your source code written in python, Java, and so on. On the other hand, we usually do not measure the coverage for an Ansible playbook. <a href="https://github.com/ks888/kirby" rel="nofollow noreferrer">Kirby</a> is the tool to support this.</p>
  
Answers: CC BY-SA 4.0
Answers: 2021-12-10T15:50:31.813Z
Answers: 2021-12-10T15:50:31.813Z
Answers: bbaassssiiee
Answers: <p>You can track configuration/infrastructure discrepancies/drift using <a href="https://dev.to/melezhik/super-flexible-infrastructure-audit-with-outthentic-2a3" rel="nofollow noreferrer">Outthentic</a>, it's easy to create a test suite to "fix" desired state and re-run it every time you need to track undesired changes. </p>
  
Answers: CC BY-SA 4.0
Answers: 2018-10-04T18:12:38.587Z
Answers: Welcome to DevOps.se Alexey.  While your tool may help the user in this case, for this to be an answer here there should be a little more to convey how this is relevant to the question.  Otherwise it looks like a link-only advertisement.
Answers: chicks
Answers: CC BY-SA 4.0
Answers: 2018-10-04T19:57:05.430Z
Answers: Hi! Sure, just given more details.
Answers: Alexey Melezhik
Answers: CC BY-SA 4.0
Answers: 2018-10-04T20:21:37.433Z
Answers: From my point of view, this does nothing more than what rundeck can do and is out of scope for this question. Specially as the answer doesn't explain how it can solve auditing a thousand servers and be alerted of a drift or present a human readable report for an auditor by giving some parseable output. From what I just read it does no more than inspec can do (for exemple) and give less usable output at scale while requiring a lot of external tools to do the job and thus being less portable.
Answers: Tensibai
Answers: CC BY-SA 4.0
Answers: 2018-10-04T20:28:15.233Z
Answers: "how it can solve auditing a thousand servers" - 
  
  I could find anything about thousands of servers in the question
Answers: Alexey Melezhik
Answers: CC BY-SA 4.0
Answers: 2018-10-04T20:30:20.357Z
Answers: "and be alerted of a drift or  present a human readable report for an auditor by giving some parseable output."
  
  neither I found this in the question. The obvious alerting of drifting is the fact that tests begin to fail ...
Answers: Alexey Melezhik
Answers: CC BY-SA 4.0
Answers: 2018-10-06T11:26:32.120Z
Answers: So you're just missing the point of what those kind of tools are for in a wider point of view than just assessing state. There's not much else to say
Answers: Tensibai
Answers: CC BY-SA 4.0
Answers: 2018-10-06T13:23:07.610Z
Answers: Not at all. You just related to the points ( scalability and report format ) that are not originaly touched in the question. However no problem with that - Outthentic is both scalable and provides sensible reports/output.
Answers: Alexey Melezhik
Answers: CC BY-SA 4.0
Answers: 2018-10-04T17:46:24.520Z
Answers: 2018-10-04T19:03:31.097Z
Answers: 2018-10-04T19:03:31.097Z
Answers: Alexey Melezhik
Answers: <p>Some options out there..</p>
  
  <p><strong>Testing tools:</strong> Sorted by github stars</p>
  
  <ul>
  <li><a href="http://serverspec.org/" rel="noreferrer">Serverspec</a> - Ruby, most popular tool out there, built on ruby's rspec</li>
  <li><a href="https://github.com/aelsabbahy/goss" rel="noreferrer">Goss</a> - YAML, simple, &lt;10MB self-contained binary, extremely fast, can generate tests from system state</li>
  <li><a href="https://github.com/chef/inspec" rel="noreferrer">Inspec</a> - Ruby, think of it as an improved serverspec, almost same syntax, made by the chef guys. Built to be easier to extend than serverspec</li>
  <li><a href="https://github.com/philpep/testinfra" rel="noreferrer">Testinfra</a> - Python, has the cool feature of being able to use Ansible's inventory/vars</li>
  </ul>
  
  <p><strong>Major differences between them:</strong></p>
  
  <p>Ultimately, I would suggest spending a day experimenting with all of them to get a feel for them before deciding for yourself.</p>
  
  <ul>
  <li>With the exception of Goss, all the frameworks can run against a remote machine (ex. over ssh). Goss only runs locally or in docker w/ dgoss.</li>
  <li>All frameworks can be run locally on a server, but require Python or Ruby to be installed or embedded. Inspec provides a self-contained &lt;150MB bundle with an embedded version of ruby. Goss is a single &lt;10MB binary with no external dependencies.</li>
  <li>Goss has built in support for nagios/sensu output, this allows for easier integration with monitoring tools.</li>
  <li>Goss tests tend to be simpler, but less flexible since it's based on YAML. Other frameworks allow you to leverage the full power of the underlying language Python/Ruby to write tests or extend the tool's functionality. (simplicity vs flexibility)</li>
  <li>Goss allows you to generate tests from current system state</li>
  <li>Testinfra to my knowledge is the only one that has built-in support for ansible inventory and variables</li>
  <li>Inspec is backed by Chef</li>
  </ul>
  
  <p><strong>Continuous/divergence testing:</strong></p>
  
  <ul>
  <li><a href="https://docs.chef.io/compliance.html" rel="noreferrer">Chef Compliance</a> - works with inspec to continuously test your servers, paid product</li>
  <li><a href="https://github.com/aelsabbahy/goss" rel="noreferrer">Goss</a> - Can be easily hooked into Nagios or Sensu. Also, supports exposing server tests as an http endpoint.</li>
  </ul>
  
  <p><strong>Testing harnesses for development:</strong></p>
  
  <ul>
  <li><a href="http://kitchen.ci/" rel="noreferrer">kitchen</a> - Testing harness tool, launches instance, runs config management code, runs test suite. Made by the chef guys</li>
  <li><a href="https://molecule.readthedocs.io" rel="noreferrer">Molecule</a> - Similar to test kitchen, but written specifically for ansible</li>
  </ul>
  
  <p><strong><em>Full Disclosure:</strong> I'm the author of goss</em></p>
  
  <p><strong>UPDATE:</strong> InSpec 4.x or above uses a mixed commercial / open source license - see comments.</p>
  
Answers: CC BY-SA 3.0
Answers: 2017-03-29T14:00:29.317Z
Answers: I understand you're a little biased, but inspec doesn't need Compliance to run periodically. And there's no dependencies to manage, all necessary libs and framework (ruby) is bundled in the package which can be installed locally on each node, when running through ssh/winrm, inspec put itself in place. You point about exclusively external command which is not true, a lot of tests are done by internal ruby code. (I think it worth being noted)
Answers: Tensibai
Answers: CC BY-SA 3.0
Answers: 2017-03-29T15:10:32.853Z
Answers: I've updated the answer to correct the external command and bundled ruby for inspec.
  
  Can you clarify or send me a link explaining how inspec can be used on it's own to detect/report on drift?
Answers: Ahmed Elsabbahy
Answers: CC BY-SA 3.0
Answers: 2017-03-29T15:20:41.467Z
Answers: Well, the reporting part will be at your hand, it's indeed Compliance goal to make a representation. But you can run inspec in a crontab and just rely on the crontab mail when there's a test failing to alert you (for example).
Answers: Tensibai
Answers: CC BY-SA 3.0
Answers: 2017-03-29T15:22:50.460Z
Answers: All in all, tahnks for the edit, that sounds a fair exposition of your tool and others. I'm afraid this may become outdated quickly, but +1 anyway for the listing and pointers.
Answers: Tensibai
Answers: CC BY-SA 3.0
Answers: 2017-03-29T15:24:18.140Z
Answers: Ah, yeah.. all tools can be leveraged in that way. I was trying to provide tools (or integrations with tools) that provide better reporting. To my knowledge there's compliance, or wrapping the tools in a way that makes them nagios/sensu/some-other-monitoring-tool friendly. Ex: https://www.slideshare.net/m_richardson/serverspec-and-sensu-testing-and-monitoring-collide
  
  Apologies if it came off as biased initially, wasn't intended to be.
Answers: Ahmed Elsabbahy
Answers: CC BY-SA 3.0
Answers: 2017-03-29T15:27:23.117Z
Answers: Indeed, all tools can be leveraged that way, it's just the sentence about compliance sounds like Compliance is necessary to run inspec periodically, which it doesn't provide, you still have to manage the inspec run to report back to compliance. You may take advantage of this reporting to send to an elasticsearch server for example, but that's hacky and out of the scope.
Answers: Tensibai
Answers: CC BY-SA 3.0
Answers: 2017-03-29T15:33:04.450Z
Answers: And no worry for the bias, I just said what I felt 'wrong' at first read. There's no problem being mistaken by the description of the tools and even less when you take it positive and review the post :)
Answers: Tensibai
Answers: CC BY-SA 4.0
Answers: 2019-10-16T10:58:04.537Z
Answers: InSpec update - current versions (4.x and higher) are now **commercial** requiring payment, though underlying source code remains open source.  If using for personal use, non-commercial use, or 'experimental' in a business, InSpec is still free, but in other cases they require a commercial license.  See [Chef licensing docs page](https://docs.chef.io/chef_license.html) and EULA.  Beware the confusing "Chef is 100% open source" statements from Apr 2019 - technically true but executable tools using Chef trademarks are subject to EULA.
Answers: RichVel
Answers: CC BY-SA 4.0
Answers: 2019-10-16T11:02:26.007Z
Answers: Commented here about InSpec license because I talked to some people using it without realising it now has a commercial license for most business usage.
Answers: RichVel
Answers: CC BY-SA 4.0
Answers: 2017-03-29T01:47:22.920Z
Answers: 2019-10-17T14:19:06.473Z
Answers: 2019-10-17T14:19:06.473Z
Answers: Ahmed Elsabbahy
Body: <p>Looking at trying to build some resilience into our Ansible setup which deals with provisioning and configuration. </p>
  
  <p>I understand a few methods of testing on the configuration side of things but I'm wondering how best to implement testing on the provisioning side of things, and if there are any tools which can help with this type of implementation.</p>
  
  <p>Currently a lot of our testing is done serially during the playbook which makes a lot of sense for stuff like "has service come up; is the vip available; has this async task finished" but what really concerns me is our ability to manage drift of configuration at both the application and provisioning layer (such as VM configuration). I'm aware Ansible isn't the best tool for working with configuration drift but I'm curious to see your own opinions.</p>
  
  <p>If you have something to fully automate the process even better. (we have a few ugly scripts which report back in slack daily).</p>
  
  <p><strong>Note</strong>: Right now we have a few conditions where a reprovision might occur (e.g. rebuild from backup, critical systems issue) but typically it just loops through some of the ansible configuring tasks and thinks no more of it.</p>
  
Comments: CC BY-SA 3.0
Comments: 2017-04-15T20:23:16.687Z
Comments: Hi @Naphta any of the answers has solved your question please consider [accepting it](https://meta.stackexchange.com/q/5234/179419) by clicking the check-mark. This indicates to the wider community that you've found a solution and gives some reputation to both the answerer and yourself. There is no obligation to do this.
Comments: Richard Slater
Comments: CC BY-SA 3.0
Comments: 2018-04-26T22:31:41.593Z
Comments: `I'm aware Ansible isn't the best tool for working with configuration drift ` Please explain.
Comments: 030
Comments: CC BY-SA 3.0
Comments: 2017-02-28T22:04:30.853Z
Comments: Does Ansible run only once upon provisioning? If not, what frequency is it being run? Just trying to understand the problem before offering a solution.
Comments: Woodland
Tags: <testing><ansible><configuration><provisioning>
Title: How to test provisioning and configuration in Ansible setup?
UserDisplayName: Naphta
ThreadText: # How to test provisioning and configuration in Ansible setup?
  
  (Asked by: username_0 on 2017-02-28)
  
  Looking at trying to build some resilience into our Ansible setup which deals with provisioning and configuration.
  
  I understand a few methods of testing on the configuration side of things but I'm wondering how best to implement testing on the provisioning side of things, and if there are any tools which can help with this type of implementation.
  
  Currently a lot of our testing is done serially during the playbook which makes a lot of sense for stuff like "has service come up; is the vip available; has this async task finished" but what really concerns me is our ability to manage drift of configuration at both the application and provisioning layer (such as VM configuration). I'm aware Ansible isn't the best tool for working with configuration drift but I'm curious to see your own opinions.
  
  If you have something to fully automate the process even better. (we have a few ugly scripts which report back in slack daily).
  
  **Note**: Right now we have a few conditions where a reprovision might occur (e.g. rebuild from backup, critical systems issue) but typically it just loops through some of the ansible configuring tasks and thinks no more of it.
  
  **Tags:** testing, ansible, configuration, provisioning
  
  > **Comments:**
  > **username_3**: Does Ansible run only once upon provisioning? If not, what frequency is it being run? Just trying to understand the problem before offering a solution.  
  > **username_1**: Hi @username_0 any of the answers has solved your question please consider [accepting it](https://meta.stackexchange.com/q/5234/179419) by clicking the check-mark. This indicates to the wider community that you've found a solution and gives some reputation to both the answerer and yourself. There is no obligation to do this.  
  > **username_2**: `I'm aware Ansible isn't the best tool for working with configuration drift ` Please explain.  
  
  ---
  
  ## 6 answers
  
  ### Answer by username_13 on 2017-03-29. Score: 24 | Accepted answer
  
  Some options out there..
  
  **Testing tools:** Sorted by github stars
  
  * [Serverspec](http://serverspec.org/) - Ruby, most popular tool out there, built on ruby's rspec
  * [Goss](https://github.com/aelsabbahy/goss) - YAML, simple, <10MB self-contained binary, extremely fast, can generate tests from system state
  * [Inspec](https://github.com/chef/inspec) - Ruby, think of it as an improved serverspec, almost same syntax, made by the chef guys. Built to be easier to extend than serverspec
  * [Testinfra](https://github.com/philpep/testinfra) - Python, has the cool feature of being able to use Ansible's inventory/vars
  
  **Major differences between them:**
  
  Ultimately, I would suggest spending a day experimenting with all of them to get a feel for them before deciding for yourself.
  
  * With the exception of Goss, all the frameworks can run against a remote machine (ex. over ssh). Goss only runs locally or in docker w/ dgoss.
  * All frameworks can be run locally on a server, but require Python or Ruby to be installed or embedded. Inspec provides a self-contained <150MB bundle with an embedded version of ruby. Goss is a single <10MB binary with no external dependencies.
  * Goss has built in support for nagios/sensu output, this allows for easier integration with monitoring tools.
  * Goss tests tend to be simpler, but less flexible since it's based on YAML. Other frameworks allow you to leverage the full power of the underlying language Python/Ruby to write tests or extend the tool's functionality. (simplicity vs flexibility)
  * Goss allows you to generate tests from current system state
  * Testinfra to my knowledge is the only one that has built-in support for ansible inventory and variables
  * Inspec is backed by Chef
  
  **Continuous/divergence testing:**
  
  * [Chef Compliance](https://docs.chef.io/compliance.html) - works with inspec to continuously test your servers, paid product
  * [Goss](https://github.com/aelsabbahy/goss) - Can be easily hooked into Nagios or Sensu. Also, supports exposing server tests as an http endpoint.
  
  **Testing harnesses for development:**
  
  * [kitchen](http://kitchen.ci/) - Testing harness tool, launches instance, runs config management code, runs test suite. Made by the chef guys
  * [Molecule](https://molecule.readthedocs.io) - Similar to test kitchen, but written specifically for ansible
  
  ***Full Disclosure:*** I'm the author of goss
  
  **UPDATE:** InSpec 4.x or above uses a mixed commercial / open source license - see comments.
  
  > **Comments:**
  > **username_12**: I understand you're a little biased, but inspec doesn't need Compliance to run periodically. And there's no dependencies to manage, all necessary libs and framework (ruby) is bundled in the package which can be installed locally on each node, when running through ssh/winrm, inspec put itself in place. You point about exclusively external command which is not true, a lot of tests are done by internal ruby code. (I think it worth being noted)  
  > **username_13**: I've updated the answer to correct the external command and bundled ruby for inspec.  Can you clarify or send me a link explaining how inspec can be used on it's own to detect/report on drift?  
  > **username_12**: Well, the reporting part will be at your hand, it's indeed Compliance goal to make a representation. But you can run inspec in a crontab and just rely on the crontab mail when there's a test failing to alert you (for example).  
  > **username_12**: All in all, tahnks for the edit, that sounds a fair exposition of your tool and others. I'm afraid this may become outdated quickly, but +1 anyway for the listing and pointers.  
  > **username_13**: Ah, yeah.. all tools can be leveraged in that way. I was trying to provide tools (or integrations with tools) that provide better reporting. To my knowledge there's compliance, or wrapping the tools in a way that makes them nagios/sensu/some-other-monitoring-tool friendly. Ex: https://www.slideshare.net/m_richardson/serverspec-and-sensu-testing-and-monitoring-collide  Apologies if it came off as biased initially, wasn't intended to be.  
  > **username_12**: Indeed, all tools can be leveraged that way, it's just the sentence about compliance sounds like Compliance is necessary to run inspec periodically, which it doesn't provide, you still have to manage the inspec run to report back to compliance. You may take advantage of this reporting to send to an elasticsearch server for example, but that's hacky and out of the scope.  
  > **username_12**: And no worry for the bias, I just said what I felt 'wrong' at first read. There's no problem being mistaken by the description of the tools and even less when you take it positive and review the post :)  
  > **username_7**: InSpec update - current versions (4.x and higher) are now **commercial** requiring payment, though underlying source code remains open source.  If using for personal use, non-commercial use, or 'experimental' in a business, InSpec is still free, but in other cases they require a commercial license.  See [Chef licensing docs page](https://docs.chef.io/chef_license.html) and EULA.  Beware the confusing "Chef is 100% open source" statements from Apr 2019 - technically true but executable tools using Chef trademarks are subject to EULA.  
  > **username_7**: Commented here about InSpec license because I talked to some people using it without realising it now has a commercial license for most business usage.  
  
  ---
  
  ### Answer by username_5 on 2017-02-28. Score: 12
  
  The two tools I've seen for this are [InSpec](https://www.chef.io/inspec/) and [ServerSpec](http://serverspec.org/). Serverspec is a Ruby-based tool that builds on [RSpec](http://rspec.info/). InSpec is inspired by RSpec and ServerSpec.
  
  I've used ServerSpec. It's cool, but maybe not 100% stable. I've had problems with testing for specific versions of software on Ubuntu.
  
  I've read the InSpec docs but haven't dug in deep. It does essentially the same thing as Serverspec.
  
  Judging by the Github commits, it looks like work on ServerSpec has tailed off somewhat, whereas InSpec is just getting ramped up.
  
  **UPDATE:** InSpec 4.x or above uses a mixed commercial / open source license - see comments.
  
  > **Comments:**
  > **username_4**: To slightly clarify, InSpec emulates RSpec but does not build on it.  
  > **username_5**: @MattSchuchard Thanks for that reference!  
  > **username_6**: well "not 100% stable" for a testing tool does not sound any good  
  > **username_7**: InSpec is very good as a test tool, and makes it easy to install nothing on the server, whereas ServerSpec can only do this with some extra work.   Some work would be needed to have InSpec use an Ansible inventory though - probably easier if inventory is in [YAML format](http://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html) (Ansible 2.4+).  
  > **username_7**: InSpec update - current versions (4.x and higher) are now **commercial** requiring payment, though underlying source code remains open source.  If using for personal use, non-commercial use, or 'experimental' in a business, InSpec is still free, but in other cases they require a commercial license.  See [Chef licensing docs page](https://docs.chef.io/chef_license.html) and EULA.  Beware the confusing "Chef is 100% open source" statements from Apr 2019 - technically true but executable tools using Chef trademarks are subject to EULA.  
  
  ---
  
  ### Answer by username_8 on 2017-03-01. Score: 11
  
  When using configuration management tools, such as Ansible, the tool itself would be responsible preventing configuration drift. Once you used Ansible to set a certain configuration, repetitive execution of Ansible will ensure your configuration is as you defined it to be. This also requires your Ansible code to be written in a manner which is idempotent.
  
  Given the above, testing provisioning can be achieved by executing your Ansible playbooks in a loop from some server. For example, a cron job, or Jenkins, can execute the playbooks every 30 minutes and report back on any failure. Having no failures means that your configuration is in check, having failures means that there is a problem getting servers into your *desired state*.
  
  In a case where you cannot trust your code to be written as idempotent, and thus you cannot really execute Ansible repetitively in a loop from an automatic server a workaround exists. You can do the same as above (run Ansible in a loop) but use its [*dry run* mode](http://docs.ansible.com/ansible/playbooks_checkmode.html). Each time Ansible reports that changes are required, the Jenkins job (or cron job) can notify you that your provisioned configuration has been altered and servers are not in their *desired state*.
  
  To make sure that your Ansible code is actually doing what you think it should be doing, the solutions mentioned by username_5 username_5 apply. Both [InSpec](https://www.chef.io/inspec/) and [Serverspec](http://serverspec.org/) are tools that verify in a *different* way that your playbooks actually does what you mean. A great way to execute these kinds of tools in test environments (even docker containers) is to use [kitchen.ci](http://kitchen.ci/) which handles all the glue between the various infra unit testing tools, and execution of your playbooks/modules/cookbooks.
  
  Kitchen.ci was originally used to test Chef cookbooks, but [plugins exist for Ansible](https://github.com/neillturner/kitchen-ansible) and other CM tools as well.
  
  
  ---
  
  ### Answer by username_4 on 2017-03-01. Score: 6
  
  Test Kitchen has a kitchen-ansible provisioner plugin for testing of Ansible code. It isn't as deep as the Chef integration but it does get the job done for most cases. There is also the more recent Molecule project which is a dedicated Ansible testing system.
  
  
  ---
  
  ### Answer by username_9 on 2021-12-10. Score: 0
  
  It is usual to measure the code coverage for your source code written in python, Java, and so on. On the other hand, we usually do not measure the coverage for an Ansible playbook. [Kirby](https://github.com/ks888/kirby) is the tool to support this.
  
  
  ---
  
  ### Answer by username_10 on 2018-10-04. Score: 0
  
  You can track configuration/infrastructure discrepancies/drift using [Outthentic](https://dev.to/username_10/super-flexible-infrastructure-audit-with-outthentic-2a3), it's easy to create a test suite to "fix" desired state and re-run it every time you need to track undesired changes.
  
  > **Comments:**
  > **username_11**: Welcome to DevOps.se username_10.  While your tool may help the user in this case, for this to be an answer here there should be a little more to convey how this is relevant to the question.  Otherwise it looks like a link-only advertisement.  
  > **username_10**: Hi! Sure, just given more details.  
  > **username_12**: From my point of view, this does nothing more than what rundeck can do and is out of scope for this question. Specially as the answer doesn't explain how it can solve auditing a thousand servers and be alerted of a drift or present a human readable report for an auditor by giving some parseable output. From what I just read it does no more than inspec can do (for exemple) and give less usable output at scale while requiring a lot of external tools to do the job and thus being less portable.  
  > **username_10**: "how it can solve auditing a thousand servers" -   I could find anything about thousands of servers in the question  
  > **username_10**: "and be alerted of a drift or  present a human readable report for an auditor by giving some parseable output."  neither I found this in the question. The obvious alerting of drifting is the fact that tests begin to fail ...  
  > **username_12**: So you're just missing the point of what those kind of tools are for in a wider point of view than just assessing state. There's not much else to say  
  > **username_10**: Not at all. You just related to the points ( scalability and report format ) that are not originaly touched in the question. However no problem with that - Outthentic is both scalable and provides sensible reports/output.  
  
  ---
  
  