Answers: <p>Cloud setup could be done completly by Terraform. Check out terraform provider resources for required parameters ex. <a href="https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/data_factory" rel="nofollow noreferrer">Azure Data Factory</a></p>
  <p>Configuration management stuff mostly in use for OS tuning inside Virtual Machines. So maybe you don't need them at all.</p>
  
Answers: CC BY-SA 4.0
Answers: 2021-01-11T13:28:22.177Z
Answers: 2021-01-11T13:28:22.177Z
Answers: Vasiliy  Shakhunov
Body: <p>I am an <em>Azure Data Engineer</em> and try to leverage the IaC benefits in this area of <em>Azure Data Factory, Azure Databricks, Azure SQL &amp; Synapse, Azure Power BI, Azure IoT</em>.
  After some research I figured out that I need two essential parts:</p>
  <pre><code>1.Provisioning part
  2.Configuration management part
  </code></pre>
  <p>Breaking down to technology I decided to use <em>Terraform</em> for the provisioning of infrastructure like (RG, Network, DBs etc).
  For the configuration management I am still not sure what to choose between <em>Chef, Puppet and Ansible</em>.
  There is another idea to use <em>Docker/Packer</em> + <em>Kubernetes</em>.</p>
  <p>What are the best practices of IaC and automation when it comes to the Area of Data Engineering/Big Data?</p>
  
Tags: <ansible><terraform><chef><infrastructure-as-code><puppet>
Title: IaC Deployment in Azure with Terraform + Chef vs. Puppet vs. Ansible for Data Engineers
UserDisplayName: Pirvu Georgian
ThreadText: # IaC Deployment in Azure with Terraform + Chef vs. Puppet vs. Ansible for Data Engineers
  
  (Asked by: username_0 on 2021-01-08)
  
  I am an *Azure Data Engineer* and try to leverage the IaC benefits in this area of *Azure Data Factory, Azure Databricks, Azure SQL & Synapse, Azure Power BI, Azure IoT*.
  After some research I figured out that I need two essential parts:
  
  ```
  1.Provisioning part
  2.Configuration management part
  
  ```
  
  Breaking down to technology I decided to use *Terraform* for the provisioning of infrastructure like (RG, Network, DBs etc).
  For the configuration management I am still not sure what to choose between *Chef, Puppet and Ansible*.
  There is another idea to use *Docker/Packer* + *Kubernetes*.
  
  What are the best practices of IaC and automation when it comes to the Area of Data Engineering/Big Data?
  
  **Tags:** ansible, terraform, chef, infrastructure-as-code, puppet
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-01-11. Score: 1
  
  Cloud setup could be done completly by Terraform. Check out terraform provider resources for required parameters ex. [Azure Data Factory](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/data_factory)
  
  Configuration management stuff mostly in use for OS tuning inside Virtual Machines. So maybe you don't need them at all.
  
  
  ---
  
  