Introduction

At SeatGeek, we manage our infrastructure with Terraform, and as part of the Developer Experience team, we’re always looking for ways to help our product teams iterate on infrastructure within their own repositories—whether it’s setting up AWS resources, configuring alerts, or spinning up a new repository. Our goal is to provide golden paths that simplify these tasks.

We know not every engineer is familiar with Terraform, and that’s why we focus on creating these golden paths: to standardize the process and lower the barrier to entry. This way, teams can get started with Terraform quickly and confidently.

To further support this effort, we’re excited to introduce two open-source packages: @seatgeek/node-hcl (Github, npm) and a Backstage plugin for HCL scaffolder actions (Github, npm), designed to make working with Terraform even easier.

🙅 The Problem: Terraform + Node.js + Backstage

The challenge we faced was finding a way to simplify the process of bootstrapping new Terraform code across our repositories, many of which already had Terraform set up. We already rely on Backstage to create software templates for Terraform, but this only worked well in the context of greenfield repositories. For repositories with existing Terraform configurations, things became a bit more complicated.

Out of the box, Backstage’s built-in fetch:plain and fetch:template actions either create new files or fully overwrite existing ones—a basically all-or-nothing approach. This presents a challenge with shared files in the root Terraform directory like provider.tf, state.tf, main.tf, variables.tf, and output.tf, which may not always be present. And when they are, overwriting them adds extra work for engineers, who then have to manually reconcile these changes–adding more engineering toil in a solution aiming to reduce it.

Since Backstage is built on Node.js, we are limited to the plugins and libraries available within that ecosystem. Initially, it appeared that existing Node.js tools could help us handle HCL. Libraries such as @cdktf/hcl2json (based on @tccmombs/hcl2json) allowed us to convert HCL to JSON, which seemed promising as the first step towards manipulating the data. Then by leveraging open-source Backstage plugins, such as Roadie’s scaffolder actions for working with JSON directly, we could have addressed part of the issue. However, without a way to convert the JSON back into HCL, this would be a partial, incomplete solution at best.

And unfortunately, converting JSON back to HCL isn’t as straightforward as it sounds. As noted in this post by Martin Atkins, while HCL can be parsed into JSON for certain use cases, it’s not without its limitations, particularly with Terraform HCL:

A tool to convert from JSON syntax to native [HCL] syntax is possible in principle, but in order to produce a valid result it is not sufficient to map directly from the JSON infoset to the native syntax infoset. Instead, the tool must map from JSON infoset to the application-specific infoset, and then back from there to the native syntax infoset. In other words, such a tool would need to be application-specific.

In other words, while it’s technically possible to convert JSON back into the native HCL format, the conversion would need to understand the specific structure and nuances of Terraform’s flavor of HCL. Otherwise, you’d end up with valid HCL syntax that isn’t valid Terraform.

This meant the only viable path was to work directly with HCL for both reading and writing—but there was no Node.js-based solution for doing exactly that as Hashicorps HCL spec is only available in Go.

That’s where we found a solution in Go’s WebAssembly port.

🔥 The Solution: Go + WebAssembly + Backstage

After exploring different options and observing how tools like HashiCorp’s terraform-cdk utilized Go-based solutions to handle converting HCL to JSON and formatting HCL in Node.js, we decided to also build on top of Golang’s WebAssembly (Wasm) capabilities. Golang’s WebAssembly port allows Go code to be compiled and run on the web, enabling JavaScript engines like Node.js to take advantage of the Go ecosystem. By leveraging this technology, we were able to write the functions we needed for merging HCL files directly in our Backstage plugins.

Here’s a quick breakdown of how we integrated Go’s WebAssembly with Node.js in our Backstage plugin to handle HCL file operations like merging:

Building @seatgeek/node-hcl

We developed @seatgeek/node-hcl to leverage Golang’s WebAssembly port as the first piece that will allow us to bridge the gap between Backstage’s Node.js-based ecosystem and the HCL world of Terraform. This package allows us to write the functions we need in Go for working directly with HCL and later export these functions as Javascript.

1. Defining the implementation: (source) We first define the Go function we need to expose, such as a Merge function that combines two HCL strings using the hclwrite package from HashiCorp’s HCL library.

2. Registering the Javascript bindings: (source) Once the Go function is defined, it is registered and made callable within JavaScript through WebAssembly. This process involves mapping the Merge function to a JavaScript-compatible format.

1
2
3
4
5
6
7
8
9

registerFn("merge", func(this js.Value, args []js.Value) (interface{}, error) {
  if len(args) < 2 {
    return nil, fmt.Errorf("Not enough arguments, expected (2)")
  }

  aHclString := args[0].String()
  bHclString := args[1].String()
  return hcl.Merge(aHclString, bHclString)
})

3. Compiling Go to Webassembly: (source, source) The final code is then compiled to WebAssembly by targeting GOOS=js GOARCH=wasm. This creates a main.wasm file that is then gzipped and packaged. Since we want to also have this executable in a browser, the Javascript support file is packaged alongside the compiled wasm.

1
2
3
4

GOOS=js GOARCH=wasm go build -o main.wasm
gzip -9 -v -c main.wasm > dist/wasm/main.wasm.gz
# ...
cp -rf "$(go env GOROOT)/misc/wasm/wasm_exec.js" dist/wasm/

4. Exporting the bindings as Javascript: (source, source) Both the compiled wasm and the Javascript support file are loaded, allowing us to wrap the WebAssembly bindings in a JavaScript function that can be imported and used within any Node.js project.

Integrating into Backstage

The last piece for putting it all together was to create a Backstage plugin that could make use of our new package. We developed @seatgeek/backstage-plugin-scaffolder-backend-module-hcl for this purpose. It provides custom scaffolder actions to read, merge, and write HCL files through Backstage software templates.

From this point, all we had to do is:

1. Import the library: Include the published @seatgeek/node-hcl package into our new Backstage plugin.

2. Register the actions: (source, source) Develop the custom scaffolder actions using Backstage’s plugin system. These actions will use the merge function we exported earlier.

3. Import the plugin: Include the published @seatgeek/backstage-plugin-scaffolder-backend-module-hcl plugin to our internal Backstage instance at packages/backend/src/index.ts.

1
2
3
4
5

const backend = createBackend();

// ...

backend.add(import('@seatgeek/backstage-plugin-scaffolder-backend-module-hcl'));

4. Reference the actions in a software template: Then with all the pieces together, anyone can just include the actions in their software templates like so:

1
2
3
4
5
6
7
8
9

spec:
  steps:
    - id: merge-hcl
      name: Merge HCL Files
      action: hcl:merge:files:write
      input:
        aSourcePath: ./a/versions.tf
        bSourcePath: ./b/versions.tf
        outputPath: ./new/versions.tf

💡 Final Thoughts

At SeatGeek, we strongly believe in the value of open source. Solving this problem wasn’t just a win for our team—it’s something we knew could help other teams in similar situations. By open-sourcing @seatgeek/node-hcl and our Backstage plugin for HCL scaffolder actions, we’re contributing back to the community that has helped us build amazing solutions. Whether you’re using Terraform, managing Backstage software templates, or simply want to work with HCL in Node.js, we hope these tools will make your life a little easier.

Both libraries are available on our GitHub. Contributions, feedback, and issues are welcome. We hope these tools can serve as a starting point or a valuable addition to your developer workflows.

Happy HCL-ing!