tl;dr: If you manage hundreds of terraform files, is it worth writing a high level engine in a fully fledged programming language to manages those files ?
I spend a lot of time, editing, copying, and changing terraform files for infrastructure changes. Although the cicd used to make changes to the infrastructure, deploying a new release, uses cp, and seds to change terraform files and create new ones.
I find that a bit complicated to introduce changes to our cicd pipelines.
I thought of having a set of template that I could manage using a python engine, to create files, plan and apply, ...
Is it a good idea ? Do you have experience, or have you tried that too ?
Generic templating of Terraform HCL modules (.tf files) seems like an almost insurmountable goal and you would lose a lot of smart editor and IDE support in the process.
You would be better off moving the changing values into .tfvars or terragrunt.hcl files and sticking to you sed script or subscribing to Gruntwork and using their terraform-update-variable script which has a lot of testing and functionality that might prove useful.
Is templating Terraform files a good idea?