Answers: <p>Packer has taken an active decision not to build in such capabilities. This problem is much better solved by other tools such as jq, etc. We think the best way to leverage Packer is to wrap it if you need more ways to create the template. </p>
  
  <p>It's very easy to create json in any programming language which gives you unlimited expressiveness when creating templates. One great way to manipulate templates is to define them in <em>jinja2</em> and preprocess it with Python.</p>
  
Answers: CC BY-SA 4.0
Answers: 2018-11-24T10:27:15.817Z
Answers: 2018-12-07T10:45:00.087Z
Answers: 2018-12-07T10:45:00.087Z
Answers: Rickard von Essen
Answers: <p>Utilizing the <a href="https://www.packer.io/docs/from-1.5" rel="nofollow noreferrer">HCL2</a> syntax is definitely the correct approach here. You can use the different building <a href="https://www.packer.io/docs/from-1.5/blocks" rel="nofollow noreferrer">blocks</a> to define different source blocks and then assign sources to the builders and even conditionally execute the provisioners and <a href="https://www.packer.io/docs/from-1.5/blocks/build/post-processors" rel="nofollow noreferrer">post processors</a> on specific sources using <a href="https://www.packer.io/docs/from-1.5/onlyexcept" rel="nofollow noreferrer">only/except</a>.</p>
  <p>Using this approach you can easily accomplish what you are looking for. Here is a rough example adapted from the hashicorp documentation with multiple sources and ansible:</p>
  <pre><code>source &quot;amazon-ebs&quot; &quot;first-example&quot; {
      ...
  }
  source &quot;amazon-ebs&quot; &quot;second-example&quot; {
      ...
  }
  build {
    name = &quot;my_build&quot;
    sources [
      &quot;source.amazon-ebs.first-example&quot;,&quot;source.amazon-ebs.second-example&quot;
    ]
  
  # I run on both sources 
  provisioner &quot;ansible&quot; {
          playbook_file = &quot;common.yml&quot;
    }
  
  # Only runs on first example
  provisioner &quot;ansible&quot; {
      only = [&quot;source.amazon-ebs.first-example&quot;]
      playbook_file = &quot;playbook-a.yml&quot;
    }
  
  # Never runs on second example
  provisioner &quot;ansible&quot; {
      except = [&quot;source.amazon-ebs.second-example&quot;]
      playbook_file = &quot;playbook-b.yml&quot;
    }
  }
  </code></pre>
  <p>Depending on your source blocks and the builder being used, this opens up the ability to template multi-cloud, arch, distribution and purposed images from the same packer build command.</p>
  
Answers: CC BY-SA 4.0
Answers: 2024-12-10T21:35:22.373Z
Answers: One thing that I'd like to figure out when doing this is for the "I run on both sources", how to specify the "user" var for example.  This is often different for each source (eg: rocky/ec2-user/ubuntu)
Answers: openCivilisation
Answers: CC BY-SA 4.0
Answers: 2024-12-11T22:15:55.137Z
Answers: The easiest way to do this is to create a variable for user, pass it into your source and then define a source specific variable file like rocky.pkvars.hcl for each source and pass it into the packer build command and orchestrate that for each source using your ci system. It also helps to debug and limit the blast radius of failures to separate them this way.
Answers: zm31
Answers: CC BY-SA 4.0
Answers: 2024-12-11T22:56:41.770Z
Answers: alternatively, you can expand on doing something like this:
  
  variable "username_map" {
    type = map(string)
    default = {
      "rocky" = "rockyuser"
      "rhel"  = "redhatuser"
    }
  }
  
  source "null" "rocky" {
    communicator = "none"
  }
  
  source "null" "rhel" {
    communicator = "none"
  }
  
  build {
    sources = [
      "source.null.rhel",
      "source.null.rocky"
    ]
  
    provisioner "shell-local" {
      inline = [
        "echo ${var.username_map[source.name]}"
      ]
    }
  }
  
  This echos "redhatuser" when running the rhel source and "rockeruser" when using that source
Answers: zm31
Answers: CC BY-SA 4.0
Answers: 2024-12-11T23:06:26.017Z
Answers: see https://developer.hashicorp.com/packer/docs/templates/hcl_templates/contextual-variables
Answers: zm31
Answers: CC BY-SA 4.0
Answers: 2024-12-12T02:08:45.993Z
Answers: That documentation is very deceptive - its not actually possible to combine sources with something like an amazon-ebs plugin, probably the only reason why it would be usefull.  You can't event add extra vars to those sources, they aren't allowd.  But what does work is using a map, to lookup values via source.name, thats about as good as it got.
Answers: openCivilisation
Answers: CC BY-SA 4.0
Answers: 2021-02-06T19:38:59.417Z
Answers: 2021-02-06T19:38:59.417Z
Answers: zm31
Answers: <p>This is what <a href="https://www.packer.io/docs/from-1.5/blocks/source" rel="nofollow noreferrer">source blocks</a> are for in HCL2. I've been using this approach recently (define builder in a source block, then use it from various build blocks, filling in the variables as needed) and believe this to be the new best practice / idiom.</p>
  
Answers: CC BY-SA 4.0
Answers: 2024-12-11T21:35:19.677Z
Answers: As far as I can tell you cannot combine source blocks with an amazon-ebs source block when I tried.
Answers: openCivilisation
Answers: CC BY-SA 4.0
Answers: 2021-02-02T09:01:35.077Z
Answers: 2021-02-02T09:01:35.077Z
Answers: timblaktu
Answers: <p>So far, I've taken to defining the default <code>builders</code> and <code>provisioners</code> we use, with <a href="https://www.packer.io/docs/templates/user-variables.html" rel="nofollow noreferrer">user variables</a> interpolated in where required, eg:</p>
  
  <pre><code> "type": "amazon-ebs",
   "instance_type": "t2.micro",
   "ami_name": "{{ user `ami-name` }} 
  </code></pre>
  
  <p>I've got files called eg. <code>packer-builders.json</code> and <code>packer-provisioners.json</code>, and then all the separate Packer templates in their own directories.</p>
  
  <p>I've then written a quick Bash script that takes the name of the template one wants to build, finds all the <code>packer-*.json</code> files, and uses <code>jq --slurp</code> to merge them in (eg. <code>jq --slurp '.[0] * .[1] * .[2]' $LIST_OF_FILES</code>) with the supplied template at the end. It saves the output and sends that along to <code>packer build</code>.</p>
  
  <p>As a bonus, I'm also using <code>jsmin</code> before hitting <code>jq</code> so that I can include comments in any of the JSON files.</p>
  
  <p>It works well, but I'm not a <em>massive</em> fan of it because it's custom and it adds an additional barrier for entry for others coming across it (much more to consider than just <code>packer build something.json</code>!</p>
  
  <p>So I'm really keen for additional answers here if there's a better way to do this.</p>
  
Answers: CC BY-SA 4.0
Answers: 2018-06-18T02:25:26.930Z
Answers: 2018-06-18T02:25:26.930Z
Answers: Tim Malone
Body: <p>I'm creating about 10-12 Packer templates which almost all work the same way. Same builder (Amazon EBS), with some small variations in AMI names, and almost the same provisioner (Ansible Remote) - sometimes with additional variables and sometimes multiple playbooks.</p>
  
  <p>But, much of the Packer templates remain the same.</p>
  
  <p>I know I can use variables to get some of the changeable values into one place in the file. But I'm keen to cut down on all of the copy-pasta in the templates.</p>
  
  <p>I could do this with a wrapper shell script utilising <code>jq</code> or something. But each complexity I add gives something else for others to learn. Try as I might, I can't seem to find a 'blessed' idiomatic way of doing this with Packer.</p>
  
  <p>Is there an idiomatic way of doing this - or at least a generally accepted way?</p>
  
Comments: CC BY-SA 4.0
Comments: 2023-07-04T08:00:21.960Z
Comments: Related issue: https://github.com/hashicorp/packer/issues/9154
Comments: stackprotector
Tags: <packer>
Title: Is there an idiomatic way to create reusable Packer templates?
UserDisplayName: Tim Malone
ThreadText: # Is there an idiomatic way to create reusable Packer templates?
  
  (Asked by: username_0 on 2018-06-17)
  
  I'm creating about 10-12 Packer templates which almost all work the same way. Same builder (Amazon EBS), with some small variations in AMI names, and almost the same provisioner (Ansible Remote) - sometimes with additional variables and sometimes multiple playbooks.
  
  But, much of the Packer templates remain the same.
  
  I know I can use variables to get some of the changeable values into one place in the file. But I'm keen to cut down on all of the copy-pasta in the templates.
  
  I could do this with a wrapper shell script utilising `jq` or something. But each complexity I add gives something else for others to learn. Try as I might, I can't seem to find a 'blessed' idiomatic way of doing this with Packer.
  
  Is there an idiomatic way of doing this - or at least a generally accepted way?
  
  **Tags:** packer
  
  > **Comments:**
  > **username_1**: Related issue: https://github.com/hashicorp/packer/issues/9154  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_2 on 2018-11-24. Score: 3
  
  Packer has taken an active decision not to build in such capabilities. This problem is much better solved by other tools such as jq, etc. We think the best way to leverage Packer is to wrap it if you need more ways to create the template.
  
  It's very easy to create json in any programming language which gives you unlimited expressiveness when creating templates. One great way to manipulate templates is to define them in *jinja2* and preprocess it with Python.
  
  
  ---
  
  ### Answer by username_3 on 2021-02-06. Score: 3
  
  Utilizing the [HCL2](https://www.packer.io/docs/from-1.5) syntax is definitely the correct approach here. You can use the different building [blocks](https://www.packer.io/docs/from-1.5/blocks) to define different source blocks and then assign sources to the builders and even conditionally execute the provisioners and [post processors](https://www.packer.io/docs/from-1.5/blocks/build/post-processors) on specific sources using [only/except](https://www.packer.io/docs/from-1.5/onlyexcept).
  
  Using this approach you can easily accomplish what you are looking for. Here is a rough example adapted from the hashicorp documentation with multiple sources and ansible:
  
  ```
  source "amazon-ebs" "first-example" {
      ...
  }
  source "amazon-ebs" "second-example" {
      ...
  }
  build {
    name = "my_build"
    sources [
      "source.amazon-ebs.first-example","source.amazon-ebs.second-example"
    ]
  
  # I run on both sources 
  provisioner "ansible" {
          playbook_file = "common.yml"
    }
  
  # Only runs on first example
  provisioner "ansible" {
      only = ["source.amazon-ebs.first-example"]
      playbook_file = "playbook-a.yml"
    }
  
  # Never runs on second example
  provisioner "ansible" {
      except = ["source.amazon-ebs.second-example"]
      playbook_file = "playbook-b.yml"
    }
  }
  
  ```
  
  Depending on your source blocks and the builder being used, this opens up the ability to template multi-cloud, arch, distribution and purposed images from the same packer build command.
  
  > **Comments:**
  > **username_4**: One thing that I'd like to figure out when doing this is for the "I run on both sources", how to specify the "user" var for example.  This is often different for each source (eg: rocky/ec2-user/ubuntu)  
  > **username_3**: The easiest way to do this is to create a variable for user, pass it into your source and then define a source specific variable file like rocky.pkvars.hcl for each source and pass it into the packer build command and orchestrate that for each source using your ci system. It also helps to debug and limit the blast radius of failures to separate them this way.  
  > **username_3**: alternatively, you can expand on doing something like this:  variable "username_map" {   type = map(string)   default = {     "rocky" = "rockyuser"     "rhel"  = "redhatuser"   } }  source "null" "rocky" {   communicator = "none" }  source "null" "rhel" {   communicator = "none" }  build {   sources = [     "source.null.rhel",     "source.null.rocky"   ]    provisioner "shell-local" {     inline = [       "echo ${var.username_map[source.name]}"     ]   } }  This echos "redhatuser" when running the rhel source and "rockeruser" when using that source  
  > **username_3**: see https://developer.hashicorp.com/packer/docs/templates/hcl_templates/contextual-variables  
  > **username_4**: That documentation is very deceptive - its not actually possible to combine sources with something like an amazon-ebs plugin, probably the only reason why it would be usefull.  You can't event add extra vars to those sources, they aren't allowd.  But what does work is using a map, to lookup values via source.name, thats about as good as it got.  
  
  ---
  
  ### Answer by username_5 on 2021-02-02. Score: 3
  
  This is what [source blocks](https://www.packer.io/docs/from-1.5/blocks/source) are for in HCL2. I've been using this approach recently (define builder in a source block, then use it from various build blocks, filling in the variables as needed) and believe this to be the new best practice / idiom.
  
  > **Comments:**
  > **username_4**: As far as I can tell you cannot combine source blocks with an amazon-ebs source block when I tried.  
  
  ---
  
  ### Answer by username_0 on 2018-06-18. Score: 3
  
  So far, I've taken to defining the default `builders` and `provisioners` we use, with [user variables](https://www.packer.io/docs/templates/user-variables.html) interpolated in where required, eg:
  
  ```
   "type": "amazon-ebs",
   "instance_type": "t2.micro",
   "ami_name": "{{ user `ami-name` }} 
  
  ```
  
  I've got files called eg. `packer-builders.json` and `packer-provisioners.json`, and then all the separate Packer templates in their own directories.
  
  I've then written a quick Bash script that takes the name of the template one wants to build, finds all the `packer-*.json` files, and uses `jq --slurp` to merge them in (eg. `jq --slurp '.[0] * .[1] * .[2]' $LIST_OF_FILES`) with the supplied template at the end. It saves the output and sends that along to `packer build`.
  
  As a bonus, I'm also using `jsmin` before hitting `jq` so that I can include comments in any of the JSON files.
  
  It works well, but I'm not a *massive* fan of it because it's custom and it adds an additional barrier for entry for others coming across it (much more to consider than just `packer build something.json`!
  
  So I'm really keen for additional answers here if there's a better way to do this.
  
  
  ---
  
  