Answers: <h1>Failure to Merge - YAML whitespace</h1>
  <p>This turned out to be because of whitespace. You can confirm this by sshing to the machine and running</p>
  <pre class="lang-sh prettyprint-override"><code>sudo grep WARNING /var/log/cloud-init.log
  </code></pre>
  <p>Which should return this,</p>
  <pre><code>util.py[WARNING]: Failed loading yaml blob. Invalid format at line 11 column 1: &quot;while scanning for the next token
  util.py[WARNING]: Failed at merging in cloud config part from part-001
  </code></pre>
  <p>Basically what happens is this,</p>
  <ol>
  <li>You upload the <code>user_data</code> to the providers instance's metadata.</li>
  <li>The client you're provisioning launches <code>cloud-init</code>.</li>
  <li>That downloads the <code>user_data</code> from the instance's metadata at <code>http://169.254.169.254/</code></li>
  <li>It sees the <code>#cloud-config</code> on line one of your <code>user_data</code></li>
  <li>You merge your own cloud_config with the downloaded <code>user_data</code></li>
  <li>The merge-algorithm doesn't normalize for whitespace, so the yaml is invalid.</li>
  <li>The merge-algorithm returns a blank cloud-config.</li>
  </ol>
  <p>You can verify this by trying to re-run <code>cloud-init init</code> which will produce something like this,</p>
  <pre><code>2022-05-16 04:50:04,351 - util.py[WARNING]: Failed loading yaml blob. Invalid format at line 11 column 1: &quot;while scanning for the next token
  found character '\t' that cannot start any token
    in &quot;&lt;unicode string&gt;&quot;, line 11, column 1:
          - podman
      ^&quot;
  2022-05-16 04:50:04,351 - util.py[WARNING]: Failed at merging in cloud config part from part-001
  </code></pre>
  <h2>Full Log</h2>
  <pre><code>__init__.py[DEBUG]: {'MIME-Version': '1.0', 'Content-Type': 'text/cloud-config', 'Content-Disposition': 'attachment; filename=&quot;part-001&quot;'}
  __init__.py[DEBUG]: Calling handler CloudConfigPartHandler: [['text/cloud-config', 'text/cloud-config-jsonp']] (text/cloud-config, part-001, 3) with frequency once-per-instance
  util.py[DEBUG]: Attempting to load yaml from string of length 252 with allowed root types (&lt;class 'dict'&gt;,)
  util.py[WARNING]: Failed loading yaml blob. Invalid format at line 11 column 1: &quot;while scanning for the next token
  found character '\t' that cannot start any token
    in &quot;&lt;unicode string&gt;&quot;, line 11, column 1:
          - podman
      ^&quot;
  util.py[WARNING]: Failed at merging in cloud config part from part-001
  util.py[DEBUG]: Failed at merging in cloud config part from part-001
  Traceback (most recent call last):
    File &quot;/usr/lib/python3/dist-packages/cloudinit/handlers/cloud_config.py&quot;, line 140, in handle_part
      self._merge_part(payload, headers)
    File &quot;/usr/lib/python3/dist-packages/cloudinit/handlers/cloud_config.py&quot;, line 116, in _merge_part
      (payload_yaml, my_mergers) = self._extract_mergers(payload, headers)
    File &quot;/usr/lib/python3/dist-packages/cloudinit/handlers/cloud_config.py&quot;, line 95, in _extract_mergers
      mergers_yaml = mergers.dict_extract_mergers(payload_yaml)
    File &quot;/usr/lib/python3/dist-packages/cloudinit/mergers/__init__.py&quot;, line 79, in dict_extract_mergers
      raw_mergers = config.pop('merge_how', None)
  AttributeError: 'NoneType' object has no attribute 'pop'
  __init__.py[DEBUG]: Calling handler CloudConfigPartHandler: [['text/cloud-config', 'text/cloud-config-jsonp']] (__end__, None, 3) with frequency once-per-instance
  util.py[DEBUG]: Writing to /var/lib/cloud/instances/i-0804054440b55ca17/cloud-config.txt - wb: [600] 26 bytes
  </code></pre>
  
Answers: CC BY-SA 4.0
Answers: 2022-05-16T04:44:16.043Z
Answers: 2022-05-16T06:04:17.403Z
Answers: 2022-05-16T06:04:17.403Z
Answers: Evan Carroll
Body: <p>I'm currently provisioning a machine with Cloud Config in <code>user_data</code> (I've also tried <code>user_data_base64</code>). When I run my command in Terraform I have,</p>
  <pre><code>resource &quot;aws_instance&quot; &quot;web&quot; {
      user_data_base64 = filebase64(&quot;${path.module}/scripts/web-init-script.yml&quot;)
  </code></pre>
  <p>However, nothing happens. It silently fails. What's the problem? The contents of <code>web-init.script.yml</code> are,</p>
  <pre class="lang-yml prettyprint-override"><code>❯ cat scripts/db-init-script.sh
  #cloud-config
  package_update: true
  package_upgrade: true
  
  fqdn: db.acme.com
  prefer_fqdn_over_hostname: true
  hostname: db
  
  packages:
      - podman
  </code></pre>
  
Tags: <terraform><yaml><cloud-init>
Title: Cloud-config in Terraform does not work, nothing seems to happen and it fails silently
UserDisplayName: Evan Carroll
ThreadText: # Cloud-config in Terraform does not work, nothing seems to happen and it fails silently
  
  (Asked by: username_0 on 2022-05-16)
  
  I'm currently provisioning a machine with Cloud Config in `user_data` (I've also tried `user_data_base64`). When I run my command in Terraform I have,
  
  ```
  resource "aws_instance" "web" {
      user_data_base64 = filebase64("${path.module}/scripts/web-init-script.yml")
  
  ```
  
  However, nothing happens. It silently fails. What's the problem? The contents of `web-init.script.yml` are,
  
  ```yml
  ❯ cat scripts/db-init-script.sh
  #cloud-config
  package_update: true
  package_upgrade: true
  
  fqdn: db.acme.com
  prefer_fqdn_over_hostname: true
  hostname: db
  
  packages:
      - podman
  
  ```
  
  **Tags:** terraform, yaml, cloud-init
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2022-05-16. Score: 0 | Accepted answer
  
  # Failure to Merge - YAML whitespace
  
  This turned out to be because of whitespace. You can confirm this by sshing to the machine and running
  
  ```sh
  sudo grep WARNING /var/log/cloud-init.log
  
  ```
  
  Which should return this,
  
  ```
  util.py[WARNING]: Failed loading yaml blob. Invalid format at line 11 column 1: "while scanning for the next token
  util.py[WARNING]: Failed at merging in cloud config part from part-001
  
  ```
  
  Basically what happens is this,
  
  1. You upload the `user_data` to the providers instance's metadata.
  2. The client you're provisioning launches `cloud-init`.
  3. That downloads the `user_data` from the instance's metadata at `http://169.254.169.254/`
  4. It sees the `#cloud-config` on line one of your `user_data`
  5. You merge your own cloud\_config with the downloaded `user_data`
  6. The merge-algorithm doesn't normalize for whitespace, so the yaml is invalid.
  7. The merge-algorithm returns a blank cloud-config.
  
  You can verify this by trying to re-run `cloud-init init` which will produce something like this,
  
  ```
  2022-05-16 04:50:04,351 - util.py[WARNING]: Failed loading yaml blob. Invalid format at line 11 column 1: "while scanning for the next token
  found character '\t' that cannot start any token
    in "<unicode string>", line 11, column 1:
          - podman
      ^"
  2022-05-16 04:50:04,351 - util.py[WARNING]: Failed at merging in cloud config part from part-001
  
  ```
  
  ## Full Log
  
  ```
  __init__.py[DEBUG]: {'MIME-Version': '1.0', 'Content-Type': 'text/cloud-config', 'Content-Disposition': 'attachment; filename="part-001"'}
  __init__.py[DEBUG]: Calling handler CloudConfigPartHandler: [['text/cloud-config', 'text/cloud-config-jsonp']] (text/cloud-config, part-001, 3) with frequency once-per-instance
  util.py[DEBUG]: Attempting to load yaml from string of length 252 with allowed root types (<class 'dict'>,)
  util.py[WARNING]: Failed loading yaml blob. Invalid format at line 11 column 1: "while scanning for the next token
  found character '\t' that cannot start any token
    in "<unicode string>", line 11, column 1:
          - podman
      ^"
  util.py[WARNING]: Failed at merging in cloud config part from part-001
  util.py[DEBUG]: Failed at merging in cloud config part from part-001
  Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/cloudinit/handlers/cloud_config.py", line 140, in handle_part
      self._merge_part(payload, headers)
    File "/usr/lib/python3/dist-packages/cloudinit/handlers/cloud_config.py", line 116, in _merge_part
      (payload_yaml, my_mergers) = self._extract_mergers(payload, headers)
    File "/usr/lib/python3/dist-packages/cloudinit/handlers/cloud_config.py", line 95, in _extract_mergers
      mergers_yaml = mergers.dict_extract_mergers(payload_yaml)
    File "/usr/lib/python3/dist-packages/cloudinit/mergers/__init__.py", line 79, in dict_extract_mergers
      raw_mergers = config.pop('merge_how', None)
  AttributeError: 'NoneType' object has no attribute 'pop'
  __init__.py[DEBUG]: Calling handler CloudConfigPartHandler: [['text/cloud-config', 'text/cloud-config-jsonp']] (__end__, None, 3) with frequency once-per-instance
  util.py[DEBUG]: Writing to /var/lib/cloud/instances/i-0804054440b55ca17/cloud-config.txt - wb: [600] 26 bytes
  
  ```
  
  
  ---
  
  