Testing IaC: Terratest & InSpec

Prove infrastructure works before prod.

A recipe can be spelled perfectly and still produce something nobody should eat. Infrastructure code works the same way. When terraform validate passes, you have learned that the file parses and every argument is the right type. You have learned nothing about whether the network you described carries traffic, whether the subnet you labelled private really is private, or whether a security group (the firewall the cloud wraps around each resource) quietly lets the entire internet reach port 22. Testing is how you find that out before an attacker does.

There are three layers, and each costs more than the one below it. Static checks read the code without touching a cloud: formatting, schema validation, linting (a fussy proofreader that knows the house style and the usual mistakes), and the security scanners from the last lesson. Seconds each. No credentials, no bill. Plan-level tests run terraform plan and assert against the values Terraform says it intends to create, so you can fail a change that opens SSH (secure shell, the encrypted remote login protocol) to the world without creating a single resource. A plan does still make the provider configure itself, so it wants credentials, but read-only ones are enough and nothing gets built. Integration tests build the thing for real in a throwaway account, poke it to see how it behaves, then tear it down. Proofread the recipe, walk the steps without lighting the stove, then cook it and taste it. Each layer answers a question the layer below it cannot.

The checks that cost you nothing

Start with what needs no cloud credentials at all. terraform init -backend=false installs the providers and modules that validate needs in order to check schemas, and skips configuring remote state, so the job runs happily in a pull request that holds no keys. The -check flag makes fmt report bad formatting instead of rewriting your branch underneath you, which is what you want in CI (continuous integration, the pipeline that runs by itself on every push). One trap catches people: fmt has a -recursive flag and validate does not. validate only ever looks at the directory you run it in, so a repo full of modules needs a loop or a wrapper, not one lucky command. Then there is tflint, which knows things the built-in validator never will: provider-specific rules, deprecated arguments, instance types that do not exist, and missing version constraints that quietly make your builds unreproducible.

terraform fmt -check -recursive
terraform init -backend=false -input=false >/dev/null && terraform validate

modules/network/main.tf
Success! The configuration is valid.

tflint --recursive

working directory: modules/network
 
1 issue(s) found:
 
Warning: Missing version constraint for provider "aws" in "required_providers" (terraform_required_providers)
 
  on main.tf line 2:
   2:   required_providers {
 
Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.6.0/docs/rules/terraform_required_providers.md

Read the exit codes, because that is all the pipeline reacts to. tflint returns 0 for a clean run, 2 when it found issues, and 1 when it could not run at all, so a broken setup and a genuine finding look different to your job. terraform fmt -check returns non-zero the moment a file needs formatting and prints the offending paths, which is that lone first line above. Any step that fails on any non-zero exit handles all of this correctly, and none of it costs a cent.

Assertions without a cloud bill

Terraform 1.6 put a test runner inside the binary, so you write the assertions in HCL (HashiCorp Configuration Language, the same language the infrastructure itself is written in) rather than learning a second language first. Test files end in .tftest.hcl and usually sit in a tests/ directory beside the module. Each run block is one scenario: set some variables, run a plan or an apply, then check conditions against resource attributes and outputs. A plan run builds nothing, which makes it cheap enough for every commit, and it is the right place to nail down the security invariants a human reviewer will eventually stop checking.

variables {
  name = "tftest"
  cidr = "10.42.0.0/16"
}
 
run "private_subnets_are_private" {
  command = plan
 
  assert {
    condition     = aws_subnet.private[0].map_public_ip_on_launch == false
    error_message = "A subnet called private must not hand out public IPs"
  }
}
 
run "no_ssh_from_the_internet" {
  command = plan
 
  assert {
    condition = length([
      for rule in aws_security_group.bastion.ingress : rule
      if rule.from_port <= 22 && rule.to_port >= 22
         && contains(rule.cidr_blocks, "0.0.0.0/0")
    ]) == 0
    error_message = "Bastion ingress must not expose SSH to 0.0.0.0/0"
  }
}

terraform init -input=false >/dev/null && terraform test

tests/network.tftest.hcl... in progress
  run "private_subnets_are_private"... pass
  run "no_ssh_from_the_internet"... fail
╷
│ Error: Test assertion failed
│
│   on tests/network.tftest.hcl line 22, in run "no_ssh_from_the_internet":
│   22:     condition = length([
│
│ Bastion ingress must not expose SSH to 0.0.0.0/0
╵
tests/network.tftest.hcl... tearing down
tests/network.tftest.hcl... fail
 
Failure! 1 passed, 1 failed.

That failure is the whole point. The module still parses, still passes the linter, and would still apply cleanly. The test is the only thing standing between a reviewer's tired Friday afternoon and a bastion (the single hardened jump host that fronts your private network) advertising SSH to every scanner on the internet. Terraform 1.7 added mock_provider, which swaps the real provider for invented values, so the entire plan-level suite runs with no cloud account and no charges, which makes it safe on a pull request from a fork you have no reason to trust. You still run terraform init first, because Terraform reads the real provider's schema in order to invent believable values, but nothing is ever sent to an API. The trade is honesty: mocked attributes are made up, so mocks prove your logic and your guardrails hold, not that AWS would accept the request.

Terratest builds it for real, then tears it down

Terratest is a Go library (Go being the compiled language Terraform itself is written in), so a test is an ordinary Go test function that drives Terraform and then asks reality what happened. Three phases, always in this order: deploy, validate, destroy. Deploy runs init and apply against a directory. Validate reads the outputs and then goes looking. It hits the load balancer over HTTP, asks the cloud API whether the subnet has a route to an internet gateway, confirms the bucket refuses an anonymous read. Destroy removes everything. Short of production this is the strongest evidence you can get, because the infrastructure genuinely existed and genuinely answered.

package test
 
import (
  "strings"
  "testing"
  "time"
 
  "github.com/gruntwork-io/terratest/modules/aws"
  http_helper "github.com/gruntwork-io/terratest/modules/http-helper"
  "github.com/gruntwork-io/terratest/modules/random"
  "github.com/gruntwork-io/terratest/modules/terraform"
  "github.com/stretchr/testify/assert"
  "github.com/stretchr/testify/require"
)
 
func TestNetworkModule(t *testing.T) {
  t.Parallel()
  region := "eu-west-1"
  name := "tt-" + strings.ToLower(random.UniqueId())   // unique per run
 
  opts := &terraform.Options{
    TerraformDir: "../modules/network",
    Vars: map[string]interface{}{
      "name": name,
      "tags": map[string]string{"owner": "terratest", "ttl": "2h"},
    },
    EnvVars: map[string]string{"AWS_REGION": region},
    NoColor: true,
  }
 
  defer terraform.Destroy(t, opts)   // queued BEFORE anything exists
  terraform.InitAndApply(t, opts)
 
  subnetID := terraform.Output(t, opts, "private_subnet_id")
  assert.NotEmpty(t, subnetID)
 
  // the security assertion: "private" must mean no route to an internet gateway
  require.False(t, aws.IsPublicSubnet(t, subnetID, region))
 
  url := terraform.Output(t, opts, "alb_url")
  http_helper.HttpGetWithRetry(t, url, nil, 200, "ok", 30, 10*time.Second)
}

The placement of defer terraform.Destroy is the entire safety story. In Go, defer queues a function to run when the surrounding function returns, including when it returns because an assertion blew up or the code panicked. Put it immediately after the options and before InitAndApply, and cleanup is already scheduled before there is anything to clean up. Move it below the assertions and the first failure walks straight past it, leaving a NAT gateway (network address translation, the managed box that lets private machines call out to the internet without being reachable from it, billed by the hour) alive in an account nobody reads. The difference between assert and require matters too. assert records the failure and carries on; require stops the test on the spot. Both still run your deferred cleanup, because a stopped test unwinds its own goroutine rather than killing the process.

cd test && go test -v -timeout 45m -run TestNetworkModule

=== RUN   TestNetworkModule
=== PAUSE TestNetworkModule
=== CONT  TestNetworkModule
TestNetworkModule 2026-07-21T09:14:02Z logger.go:66: Running command terraform with args [init -upgrade=false]
TestNetworkModule 2026-07-21T09:14:31Z logger.go:66: Running command terraform with args [apply -input=false -auto-approve -var name=tt-a7kq2p ...]
TestNetworkModule 2026-07-21T09:17:52Z logger.go:66: Apply complete! Resources: 14 added, 0 changed, 0 destroyed.
TestNetworkModule 2026-07-21T09:17:53Z logger.go:66: Running command terraform with args [output -no-color -json private_subnet_id]
    network_test.go:38:
        	Error Trace:	/home/dev/infra/test/network_test.go:38
        	Error:      	Should be false, but was true
        	Test:       	TestNetworkModule
TestNetworkModule 2026-07-21T09:18:05Z logger.go:66: Running command terraform with args [destroy -auto-approve -input=false ...]
TestNetworkModule 2026-07-21T09:21:40Z logger.go:66: Destroy complete! Resources: 14 destroyed.
--- FAIL: TestNetworkModule (458.11s)
FAIL
exit status 1
FAIL	github.com/acme/infra/test	458.29s

Read that transcript as an operator, not as a developer. The assertion caught a subnet that was routable to the internet even though the module called it private. The test failed loudly. Destroy still ran, so the finding cost you seven and a half minutes and a few cents instead of a live exposure and an incident channel. And because every resource carried a unique name from random.UniqueId() plus an owner = terratest tag, on the day teardown does fail you can find and kill the leftovers by tag rather than by archaeology.

InSpec questions the machine that is already running

Terratest proves your module can build working infrastructure. It says nothing about the host that has been running for eight months since, drifting quietly while people patched it, debugged it, and forgot. InSpec is the health inspector who turns up at the finished kitchen and opens the fridge. It runs a profile (a versioned bundle of controls, pinned and shipped like any other dependency) against a live target and reports pass or fail on each one. Controls are written in a Ruby DSL (Ruby is a general-purpose programming language; a DSL, or domain-specific language, is a small vocabulary shaped for exactly one job) built out of resources like port, sshd_config and systemd_service, matchers like be_listening, and metadata: an id, a title, and an impact from 0.0 to 1.0 that reports and CI gates sort on. Use the desc line to record where the control came from, usually a CIS number (Center for Internet Security, whose hardening benchmarks most baselines are copied from), because the auditor will ask.

control 'net-02' do
  impact 1.0
  title 'SSH must not be reachable from the internet'
  desc 'CIS 5.2 / SEC-14: sshd binds to the management interface only.'
  describe port(22) do
    it { should be_listening }
    its('addresses') { should_not include '0.0.0.0' }
    its('addresses') { should_not include '::' }
  end
end
 
control 'ssh-03' do
  impact 1.0
  title 'SSH must not accept passwords'
  describe sshd_config do
    its('PermitRootLogin')        { should eq 'no' }
    its('PasswordAuthentication') { should eq 'no' }
  end
end
 
control 'svc-04' do
  impact 0.7
  title 'auditd must be running and enabled'
  describe systemd_service('auditd') do
    it { should be_installed }
    it { should be_enabled }
    it { should be_running }
  end
end

That second address check is not padding. A daemon bound to the IPv6 wildcard :: is reachable from the whole internet on any host with a public IPv6 address, and a control that only looks at 0.0.0.0 will cheerfully tell you everything is fine. Now point the profile at something real. -t ssh://user@host questions a live server, -t docker://<container id> runs the same controls inside a running container, and the cloud resource packs (extra bundles of resources for AWS, Azure and GCP) check an account's API state instead of a machine. --reporter cli json:path writes the human view to your terminal and machine-readable evidence to a file in the same run, in JSON (JavaScript Object Notation, the plain-text format machines parse without guessing), which is the artifact an auditor actually wants. One thing bites everyone once: Chef InSpec 4 and later refuse to start until you accept the licence, so put CHEF_LICENSE=accept-no-persist in the environment. Miss it and the first run waits for a prompt nobody will ever answer, which in a pipeline is indistinguishable from a hang.

CHEF_LICENSE=accept-no-persist inspec exec profiles/web-baseline \
  -t ssh://[email protected] -i ~/.ssh/inspec_ed25519 \
  --reporter cli json:evidence/web1-2026-07-21.json

Profile:   Web host baseline (web-baseline)
Version:   1.2.0
Target:    ssh://[email protected]:22
 
  ✔  net-02: SSH must not be reachable from the internet
     ✔  Port 22 is expected to be listening
     ✔  Port 22 addresses is expected not to include "0.0.0.0"
     ✔  Port 22 addresses is expected not to include "::"
  ×  ssh-03: SSH must not accept passwords (1 failed)
     ✔  SSHD Configuration PermitRootLogin is expected to eq "no"
     ×  SSHD Configuration PasswordAuthentication is expected to eq "no"
     expected: "no"
          got: "yes"
     (compared using ==)
  ✔  svc-04: auditd must be running and enabled
     ✔  Systemd Service auditd is expected to be installed
     ✔  Systemd Service auditd is expected to be enabled
     ✔  Systemd Service auditd is expected to be running
 
Profile Summary: 2 successful controls, 1 control failure, 0 controls skipped
Test Summary: 7 successful, 1 failure, 0 skipped

The exit code carries the verdict, and automation should read it: 0 when everything passed, 100 when at least one test failed, 101 when tests were skipped and nothing failed, 1 for a usage error. That 101 exists because of waivers. A control you have formally accepted the risk on, listed in a waiver file with run: false, a justification and an expiration_date, reports as skipped instead of failed. Treating every non-zero code as a hard failure is a defensible default, but 101 deserves a read rather than a shrug, because a skipped control is a control that told you nothing. The date does real work: once it passes, InSpec ignores the waiver and the control goes back to failing on its own. And if your organisation would rather not sign Chef's licence at all, cinc-auditor is the same code built without the trademarks, and it takes identical arguments.

Keep asking, on a schedule

A compliance check you run by hand during an audit is theatre. Run it on a timer and it becomes detection. systemd (the program that starts and supervises services on nearly every modern Linux distribution) does this with two small files: a service that runs the scan and exits, and a timer that decides when. Type=oneshot is the run-it-and-be-done flavour, as opposed to a web server that is expected to stay up forever. A timer with no Unit= line activates the service of the same name, which is why the pair below share inspec-baseline.

[Unit]
Description=Nightly InSpec baseline scan
Wants=network-online.target
After=network-online.target
 
[Service]
Type=oneshot
Environment=CHEF_LICENSE=accept-no-persist
Environment=INSPEC_CONFIG_DIR=/var/lib/inspec
ExecStart=/usr/bin/inspec exec /opt/profiles/web-baseline \
    --waiver-file /opt/profiles/waivers.yaml \
    --reporter cli json:/var/lib/inspec/baseline.json
SuccessExitStatus=101
StateDirectory=inspec
ProtectSystem=strict
ProtectHome=read-only
PrivateTmp=yes
NoNewPrivileges=yes

The bottom half of that unit is a locked room with one open drawer. ProtectSystem=strict makes the whole filesystem read-only to this service, and StateDirectory=inspec creates /var/lib/inspec on first start, hands it to the service, and makes it the single writable path, so you never have to remember to mkdir it yourself. ProtectHome=read-only closes off /root and /home, which would otherwise be where InSpec tries to write its own scratch directory, so INSPEC_CONFIG_DIR redirects that into the drawer you did open. SuccessExitStatus=101 is the line that keeps the alert meaningful, and it deserves its own explanation below.

[Unit]
Description=Run the InSpec baseline scan nightly
 
[Timer]
OnCalendar=*-*-* 02:30:00
RandomizedDelaySec=30m
FixedRandomDelay=yes
Persistent=true
 
[Install]
WantedBy=timers.target

sudo systemctl daemon-reload
sudo systemctl enable --now inspec-baseline.timer
systemctl list-timers inspec-baseline.timer

Created symlink /etc/systemd/system/timers.target.wants/inspec-baseline.timer → /etc/systemd/system/inspec-baseline.timer.
NEXT                        LEFT     LAST PASSED UNIT                  ACTIVATES
Wed 2026-07-22 02:41:07 UTC 15h left -    -      inspec-baseline.timer inspec-baseline.service
 
1 timers listed.
Pass --all to see loaded but inactive timers, too.

Four settings there earn their keep. Persistent=true runs the missed scan at boot if the machine was switched off at half past two, so a laptop or a spot instance does not silently skip a night. RandomizedDelaySec=30m stops five hundred hosts scanning in the same second and flattening whatever they report to, and FixedRandomDelay=yes derives that offset from the machine ID so each host keeps its own slot instead of wandering every night. That randomisation is baked into what you just saw: the NEXT column reads 02:41:07, not a round 02:30, because systemd shows the time it will actually fire. SuccessExitStatus=101 tells systemd that skipped-but-nothing-failed is a healthy outcome, because without it a single waived control marks the unit failed and trains everyone to ignore the alert. Real failures you want extremely visible.

systemctl status inspec-baseline.service --no-pager

× inspec-baseline.service - Nightly InSpec baseline scan
     Loaded: loaded (/etc/systemd/system/inspec-baseline.service; static)
     Active: failed (Result: exit-code) since Wed 2026-07-22 02:41:19 UTC; 6h ago
TriggeredBy: ● inspec-baseline.timer
    Process: 8123 ExecStart=/usr/bin/inspec exec /opt/profiles/web-baseline --waiver-file /opt/profiles/waivers.yaml --reporter cli json:/var/lib/inspec/baseline.json (code=exited, status=100)
   Main PID: 8123 (code=exited, status=100)
        CPU: 11.402s
 
Jul 22 02:41:07 web1 systemd[1]: Starting Nightly InSpec baseline scan...
Jul 22 02:41:19 web1 inspec[8123]:   ×  ssh-03: SSH must not accept passwords (1 failed)
Jul 22 02:41:19 web1 inspec[8123]: Profile Summary: 2 successful controls, 1 control failure, 0 controls skipped
Jul 22 02:41:19 web1 systemd[1]: inspec-baseline.service: Main process exited, code=exited, status=100/n/a
Jul 22 02:41:19 web1 systemd[1]: inspec-baseline.service: Failed with result 'exit-code'.
Jul 22 02:41:19 web1 systemd[1]: Failed to start Nightly InSpec baseline scan.

That is the defender's payoff, and it needed no new plumbing. The unit sits in a failed state carrying InSpec's own exit code, so systemctl --failed lists it, the systemd collector in the metrics agent you already run exports it, and the journal already holds the name of the failing control. Somebody re-enabled password authentication on a web host at three in the morning, whether that was a rushed fix or an intruder restoring an easier way back in, and by breakfast it is an alert with a timestamped JSON file beside it naming the control, the impact, and the value it actually found.

Where to spend the money

Most teams run the cheap layers on everything and reserve the expensive ones for blast radius. fmt, validate, tflint, the scanners and the plan-level tests go on every commit in every repository, because they cost nothing and nobody has to make a judgement call about whether to bother. Terratest goes on the shared modules that dozens of teams import, and it runs before you tag a release rather than on every push, because a flaw in a module everyone consumes is a flaw in everyone's production. InSpec goes on the golden image build (the one base image every server is stamped out of) and on the running fleet, because that is where reality slowly parts company with the code. You do not need every layer everywhere. You need the cheap ones always, and the real ones wherever a mistake would be expensive to undo.

Pick the one module every team in your company imports, the network or the account baseline, and write a single Terratest that asserts the security property everybody already assumes it has: no public route on the private subnets, encryption on by default, no wildcard in the IAM policy (identity and access management, the rules deciding who may do what). Run it in the sandbox account before your next release. One real test on the module with the most consumers buys more safety than twenty spread across modules nobody depends on.

Try this

Run terraform fmt -check -recursive on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: a killed test never cleans up after itself. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related