Terratest in CI/CD

Credentials, cleanup, gating.

Advanced12 min · lesson 11 of 12

Running Terratest in CI gives you the payoff — every module change is proven to actually work before merge — but it needs care because the CI job now deploys real infrastructure. The essentials: give the job short-lived, sandbox-scoped cloud credentials via OIDC (never long-lived keys, never production access), set generous timeouts, ensure cleanup runs even on failure or cancellation, and gate it appropriately (module PRs run the suite; not every unrelated commit needs it).

.github/workflows/terratest.yml

jobs:
  terratest:
    permissions: { id-token: write, contents: read }   # OIDC
    steps:
      - uses: aws-actions/configure-aws-credentials@v4
        with: { role-to-assume: arn:aws:iam::SANDBOX:role/terratest, aws-region: us-east-1 }
      - run: cd test && go test -v -timeout 45m ./...
      # a scheduled sweep job reaps anything a failed run leaked

Reliability and gating

Two more practical concerns. Reliability: infra tests can flake on cloud hiccups, so build in retries at the assertion level (not test-level reruns that mask real bugs), and keep tests isolated so a flake in one does not cascade. Gating: because the suite is slow and costly, many teams run the full Terratest suite on module PRs and nightly, while fast static checks gate every commit — a pragmatic split that keeps the pyramid’s economics working in CI.

A cancelled CI job can skip cleanup — sweep as backstop

If a CI run is cancelled or the runner dies mid-test, the deferred destroy may never execute, leaking resources despite your best in-test cleanup. Never rely solely on defer-destroy in CI: pair it with unique test tags and a scheduled sweep (cloud-nuke) in the sandbox account that reaps anything old, so an interrupted job cannot leave infrastructure billing indefinitely. Assume cleanup will sometimes be skipped and design for it.

Cost, safety & isolationThe IaC testing landscape