Courses Terraform security and complianceScanning & policy as code

Policy gates in the pipeline

conftest against the plan, exit codes, and blocking the merge.

Advanced25 min · lesson 8 of 12

A bouncer who checks your identification, sees that it is fake, writes a note about it, and waves you through is not doing security. He is doing paperwork. Plenty of infrastructure scanners work exactly like that bouncer: findings print in yellow, the pipeline goes green, and by week three nobody opens the log. A gate is the other thing. A gate is a check whose result decides whether a change is allowed to continue. This lesson turns the OPA (Open Policy Agent) policies from the previous lesson into a gate that keeps a public S3 (Simple Storage Service) bucket out of an AWS (Amazon Web Services) account, and that nobody can quietly step around.

The whole mechanism is one number

A delivery driver gets back to the depot and files one thing: delivered, or not delivered. Every command-line program ends the same way, by handing back a single number called the exit code. Zero means the work was done. Anything from 1 to 255 means trouble. That number is the entire contract between your policy tool and your CI (continuous integration) system, the machine that runs your builds. GitHub Actions, GitLab CI, and Jenkins all behave identically here: run the command, read the number, mark the step red if it is not zero, stop the job. There is no policy integration to install, no plugin, no webhook. If your checker exits non-zero when it finds a violation, you already have a gate.

So the first question to ask about any scanner is what it does with that number, because the defaults differ and the defaults bite. Conftest exits 1 when a `deny` rule fires. Checkov exits 1 on a failed check unless you pass `--soft-fail`. Trivy, which absorbed tfsec, prints its findings and still exits 0 unless you pass `--exit-code 1`. A team can run Trivy on every build for a year, watch it list HIGH severity misconfigurations each time, and never once fail a pipeline. Run your gate command by hand and look at `echo $?` before you trust a green tick.

Give the gate something real to read

Point the policy at the plan, not at the `.tf` files. Those files are the architect's drawing, full of "same as the other one" and "whatever the standard is around here". The plan is the builder's itemised order: the exact set of resources Terraform intends to create, change, or destroy, with variables resolved, modules expanded, and provider defaults filled in. HCL (HashiCorp Configuration Language), the language the `.tf` files are written in, hides all of that behind variables and module calls, so a policy that reads HCL is guessing. Terraform will hand you the finished plan as JSON (JavaScript Object Notation), a plain-text data format any tool can parse, and that JSON is what the policy engine reads.

terminal

# save the plan to a file, then convert that file to JSON
terraform plan -input=false -out=tfplan.binary
terraform show -json tfplan.binary > tfplan.json

output

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create
 
Terraform will perform the following actions:
 
  # aws_s3_bucket.assets will be created
  + resource "aws_s3_bucket" "assets" {
      + bucket = "acme-prod-assets"
      + tags   = {
          + "owner" = "platform"
        }
      ...
    }
 
  # aws_s3_bucket_public_access_block.assets will be created
  + resource "aws_s3_bucket_public_access_block" "assets" {
      + block_public_acls       = false
      + block_public_policy     = false
      + bucket                  = (known after apply)
      + id                      = (known after apply)
      + ignore_public_acls      = false
      + restrict_public_buckets = false
    }
 
Plan: 2 to add, 0 to change, 0 to destroy.
 
Saved the plan to: tfplan.binary
 
To perform exactly these actions, run the following command to apply:
    terraform apply "tfplan.binary"

terminal

# jq is a command-line tool for picking JSON apart. This is all the policy
# ever sees: an array of resource changes.
jq '.resource_changes[]
    | select(.type == "aws_s3_bucket_public_access_block")
    | {address, actions: .change.actions, after: .change.after}' tfplan.json

output

{
  "address": "aws_s3_bucket_public_access_block.assets",
  "actions": [
    "create"
  ],
  "after": {
    "block_public_acls": false,
    "block_public_policy": false,
    "ignore_public_acls": false,
    "restrict_public_buckets": false
  }
}

Two fields are missing from `after`, and the reason matters. `bucket` and `id` are not known until apply time, so Terraform files them under a separate `after_unknown` object instead. Remember that shape, because it decides how your rule behaves when it cannot see a value, and that is the interesting case.

One deny, one warn, and the difference between them

main.tf

resource "aws_s3_bucket" "assets" {
  bucket = "acme-prod-assets"
 
  tags = {
    owner = "platform"
  }
}
 
# S3 has blocked public access on new buckets by default since April 2023,
# so opening one up takes deliberate effort. This is what that effort
# looks like when somebody writes it down.
resource "aws_s3_bucket_public_access_block" "assets" {
  bucket = aws_s3_bucket.assets.id
 
  block_public_acls       = false
  block_public_policy     = false
  ignore_public_acls      = false
  restrict_public_buckets = false
}

Those four settings are the safety net under the trapeze, and flipping them to `false` deserves a precise description. The change publishes nothing by itself. What it does is take the net away, so that the next bucket policy with a `Principal` of `*`, the next `public-read` ACL (access control list) applied by some other module, or the next hurried click in the console takes effect instead of being ignored. The exposure arrives later, in a change nobody connects to this one, and the person who finds it is usually a stranger running a bucket-name scanner. That is why this is a `deny` and not a note in the review.

policy.rego

package main
 
import rego.v1
 
# A bucket counts as locked down only when all four switches are on.
locked_down(after) if {
	after.block_public_acls
	after.block_public_policy
	after.ignore_public_acls
	after.restrict_public_buckets
}
 
# deny = stop the pipeline
deny contains msg if {
	some r in input.resource_changes
	r.type == "aws_s3_bucket_public_access_block"
	r.change.after != null # ignore pure deletions
	not locked_down(r.change.after)
	msg := sprintf(
		"%s: public access is not blocked (all four block_public_* settings must be true)",
		[r.address],
	)
}
 
# warn = tell someone, keep going
warn contains msg if {
	some r in input.resource_changes
	r.type == "aws_s3_bucket"
	r.change.after != null
	not r.change.after.tags.owner
	msg := sprintf("%s: no owner tag, so nobody knows who to call at 3am", [r.address])
}

Rego is the language OPA policies are written in, and its version matters here. Conftest releases from 2025 onwards embed OPA v1, where the `if` and `contains` keywords are required rather than optional; `import rego.v1` does nothing on those builds and keeps the same file working on older ones. Now look at what the deny rule does when it cannot see a setting. If `block_public_acls` came from an expression Terraform cannot resolve at plan time, the key lands in `after_unknown` and is absent from `after`, `locked_down` is undefined, `not locked_down(...)` is therefore true, and the rule fires. A gate that cannot read a value should assume the worst.

Watch it block, then watch it pass

terminal

conftest test tfplan.json --policy policy/
echo "exit=$?"

output

FAIL - tfplan.json - main - aws_s3_bucket_public_access_block.assets: public access is not blocked (all four block_public_* settings must be true)
 
2 tests, 1 passed, 0 warnings, 1 failure, 0 exceptions
exit=1

Read that line left to right: the file checked, the Rego package the rule lives in (`main`, which is where conftest looks unless you tell it otherwise), then your message. The count says two tests because the policy file holds two rule bodies, one `deny` and one `warn`. Conftest counts every body it evaluated that produced no message as a pass, and this bucket carries an owner tag, so the warn body passed. Everything above `exit=1` is written for humans. The `1` is the part CI reads.

main.tf

resource "aws_s3_bucket_public_access_block" "assets" {
  bucket = aws_s3_bucket.assets.id
 
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

terminal

# the same three commands after flipping the settings (plan output omitted)
terraform plan -input=false -out=tfplan.binary
terraform show -json tfplan.binary > tfplan.json
conftest test tfplan.json --policy policy/
echo "exit=$?"

output

2 tests, 2 passed, 0 warnings, 0 failures, 0 exceptions
exit=0

Now the other half. Here is the identical gate run against a plan where the bucket is locked down properly but carries no owner tag, which is the kind of thing you want people told about without stopping their work.

terminal

conftest test tfplan-untagged.json --policy policy/
echo "exit=$?"

output

WARN - tfplan-untagged.json - main - aws_s3_bucket.assets: no owner tag, so nobody knows who to call at 3am
 
2 tests, 1 passed, 1 warning, 0 failures, 0 exceptions
exit=0

Same tool, same run, same shape of report, completely different consequence. Use `deny` where the violation is unsafe and the fix is obvious: public storage, an unencrypted volume, port 22 (the door SSH, or secure shell, knocks on) open to 0.0.0.0/0, which means every address on the internet. Use `warn` for two situations. The first is a rule that is usually right but has honest exceptions, tagging conventions being the classic. The second is any brand new rule at all: ship it as `warn`, count how often it fires over a fortnight, fix the noise, then promote it. A rule that blocks a third of pull requests on its first day teaches an entire team that policy is an obstacle to be routed around. Once the noise is gone, `conftest test --fail-on-warn` turns every warning into a build failure without a single rule being rewritten, and it grades them: warnings exit 1, failures exit 2, and CI only cares that the number is not zero.

terminal

# conftest wraps OPA. Plain opa eval needs --fail-defined, because on its own
# it prints a page of violations and still exits 0.
opa eval --fail-defined --format pretty \
  --data policy/ --input tfplan.json 'data.main.deny[_]'
echo "exit=$?"

output

"aws_s3_bucket_public_access_block.assets: public access is not blocked (all four block_public_* settings must be true)"
exit=1

Wire it into the pipeline

.github/workflows/tf.yml

name: terraform
 
on:
  pull_request:
  push:
    branches: [main]
 
permissions:
  contents: read
  id-token: write          # lets the job mint an OIDC token to swap for AWS creds
 
jobs:
  policy-gate:             # this job name becomes the required status check
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
 
      - uses: aws-actions/configure-aws-credentials@v6
        with:
          role-to-assume: arn:aws:iam::111122223333:role/tf-plan-readonly
          aws-region: eu-west-1
 
      - uses: hashicorp/setup-terraform@v4
        with:
          terraform_version: 1.15.8
          terraform_wrapper: false      # the wrapper garbles JSON on stdout
 
      # -lockfile=readonly: fail rather than quietly rewrite .terraform.lock.hcl,
      # which records the provider versions and the checksums they must match.
      - run: terraform init -input=false -lockfile=readonly
      - run: terraform plan -input=false -out=tfplan.binary
      - run: terraform show -json tfplan.binary > tfplan.json
 
      - name: Install conftest
        run: |
          VER=0.68.2                    # pin it, and check it against checksums.txt
          curl -sSfL "https://github.com/open-policy-agent/conftest/releases/download/v${VER}/conftest_${VER}_Linux_x86_64.tar.gz" \
            | sudo tar -xzf - -C /usr/local/bin conftest
 
      - name: Policy gate
        run: conftest test tfplan.json --policy policy/
        # no continue-on-error, no "|| true", no piping into tee
 
      # Apply comes last, and only on main. The plan role cannot write, so swap
      # to the apply role first. That role's trust policy pins the OIDC sub claim
      # to repo:acme/infra:ref:refs/heads/main. A pull request run presents
      # repo:acme/infra:pull_request instead, and AWS refuses it.
      - uses: aws-actions/configure-aws-credentials@v6
        if: github.event_name == 'push' && github.ref == 'refs/heads/main'
        with:
          role-to-assume: arn:aws:iam::111122223333:role/tf-apply
          aws-region: eu-west-1
 
      - name: Apply
        if: github.event_name == 'push' && github.ref == 'refs/heads/main'
        run: terraform apply -input=false tfplan.binary  # the gated plan, byte for byte

Everything before the gate builds the evidence, and everything after it runs only because the gate returned 0. The plan uses a role obtained through OIDC (OpenID Connect), a handshake where GitHub proves which workflow is asking so AWS can hand back credentials that live for minutes, instead of an access key sitting in a secret for three years. That role can describe your infrastructure but cannot create, change, or destroy any of it, so a hostile pull request that reaches this job has nothing to gain, even if the same pull request edited the workflow file. It still needs to read the state file and take the state lock, so "read-only" means read-only where it counts, not literally nothing. One more line earns its keep: `terraform_wrapper: false`. The setup action normally wraps the Terraform binary to capture its output as step outputs, and that wrapping corrupts the JSON you are redirecting into a file. Here is what the gate step prints when someone opens the pull request from the top of this lesson.

output

Run conftest test tfplan.json --policy policy/
FAIL - tfplan.json - main - aws_s3_bucket_public_access_block.assets: public access is not blocked (all four block_public_* settings must be true)
 
2 tests, 1 passed, 0 warnings, 1 failure, 0 exceptions
Error: Process completed with exit code 1.

That last line is GitHub Actions doing the only thing it needs to do. The step failed, so the job failed, so the apply step never ran, so the bucket does not exist. GitLab CI behaves the same way: a script line that exits non-zero fails the job, and a failed job fails the stage.

Where the gate sits

1Pull request opened

CI runs terraform plan with a role that can read but not write

2Plan becomes JSON

terraform show -json tfplan.binary > tfplan.json

3conftest test

a deny message exits 1, warnings on their own exit 0

4Required status check

non-zero exit turns the check red and kills the merge button

5Apply the gated plan

terraform apply tfplan.binary, the same bytes that were checked

The gate belongs between plan and apply. A policy check that runs after apply is a report about damage, not a control.

Gate before apply, and apply the plan you gated

Two ordering mistakes turn a working gate into decoration. The first is running the policy job after `terraform apply`, where it reports faithfully on a public bucket that now exists. The second is subtler: gating `tfplan.json` and then running a bare `terraform apply -auto-approve`, which throws the checked plan away and computes a fresh one. In the gap, another change merges, a data source resolves differently, or a floating module version moves, and you apply something no policy ever read. Hand the saved plan file to apply so the thing checked and the thing built are identical. State locking on the backend, such as the S3 backend's `use_lockfile` option, stops two applies colliding, but it does nothing about applying a plan nobody checked. One caveat on the plan file itself: it holds resolved variable values in cleartext, and so does the JSON you generate from it. Marking a variable `sensitive = true` only hides it from console output; it encrypts nothing, in state or in a plan. Keep both files inside the job, and if you must pass them between jobs, keep the artifact private and the retention short.

Make the check required, or it is only advice

A red check on a pull request blocks nothing by default. It is a sign on a door, not a lock. The merge button still works, and someone in a hurry will use it. The lock comes from telling the platform that this specific check must be green. On GitHub, open Settings, then Branches, add a branch protection rule for `main` (or a ruleset, which does the same job with better scoping), enable "Require status checks to pass before merging", and add `policy-gate` by name. That name is the job name from the workflow file, so renaming the job quietly unrequires the check and nobody gets an email about it. Tick "Do not allow bypassing the above settings" as well, or administrators keep an override, and administrators tend to be the people under the most deadline pressure. GitLab's equivalent is "Pipelines must succeed" in the merge request settings. With that in place, the pull request page reads:

output

Some checks were not successful
1 failing check
 
  X   terraform / policy-gate       Failing after 1m 12s      Details
 
Merging is blocked
Required statuses must pass before merging

Push the fix to the same branch and the job log and the merge box change together.

terminal

# flip the four settings to true, then push to the same pull request branch
git commit -am "s3: block public access on the assets bucket"
git push

output

[fix/s3-public-access 4d7e8b0] s3: block public access on the assets bucket
 1 file changed, 4 insertions(+), 4 deletions(-)
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 8 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 412 bytes | 412.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0), pack-reused 0
To github.com:acme/infra.git
   9f1c2ab..4d7e8b0  fix/s3-public-access -> fix/s3-public-access

The push starts the workflow again. This time the gate prints two passes and exits 0, the job goes green, and the box at the bottom of the pull request swaps sides.

output

All checks have passed
1 successful check
 
  ok  terraform / policy-gate       Successful in 1m 05s
 
This branch has no conflicts with the base branch
Merging can be performed automatically

Five quiet ways a gate stops gating

Every one of these produces a green tick over a check that found something. (1) `continue-on-error: true` on the step or the job, which reports success whatever the command returned. (2) A trailing `|| true`, the same trick typed by hand. (3) Piping the gate somewhere, as in `conftest test tfplan.json | tee report.txt`. A pipeline's exit code is the last command's, and the default shell for a `run:` step on Linux is `bash -e` without `pipefail`, so `tee` returns 0 and the failure evaporates; writing `shell: bash` explicitly gets you `-eo pipefail` and closes that one. (4) A scanner left on its permissive default, such as Trivy without `--exit-code 1` or Checkov with `--soft-fail`. (5) Rules written as `warn` that were meant to be `deny`. Test the control the way you test a smoke alarm rather than trusting the little green light: push a branch that deliberately sets `block_public_acls = false`, confirm the check goes red and the merge button dies, then delete the branch. Do it again after every edit to the workflow file.

Break glass in the open

Sooner or later a real deadline meets a real rule, and someone needs the gate to let a change through. Every building has a little box with a hammer beside the fire door, because the alternative is people propping the door open permanently. Policy needs the same box. If the only available move is commenting out the policy step, that is the move somebody makes at 11pm, and nothing about it will be visible next month. So give people a documented path that is narrower than the one they would improvise. A break-glass exception should name the exact resource, carry a ticket and an owner, expire on a date, and live in the repository, so that granting one is itself a reviewed pull request.

policy.rego

package main
 
import rego.v1
 
# The break-glass register. One entry per exception, in git, reviewed like any
# other change. No entry, no exception.
break_glass := [{
	"address": "aws_s3_bucket_public_access_block.marketing_site",
	"ticket": "SEC-4412",
	"owner": "@dana",
	"reason": "launch week static site; CloudFront migration tracked in SEC-4413",
	"expires": "2026-08-04T00:00:00Z", # 14 days maximum, no renewals by default
}]
 
locked_down(after) if {
	after.block_public_acls
	after.block_public_policy
	after.ignore_public_acls
	after.restrict_public_buckets
}
 
live(e) if time.parse_rfc3339_ns(e.expires) > time.now_ns()
 
excused(address) if {
	some e in break_glass
	e.address == address
	live(e)
}
 
# The deny rule from earlier with one line added. Edit it in place; keep a
# second copy and the old rule fires straight through the exception.
deny contains msg if {
	some r in input.resource_changes
	r.type == "aws_s3_bucket_public_access_block"
	r.change.after != null
	not locked_down(r.change.after)
	not excused(r.address) # the only escape hatch, and it is in version control
	msg := sprintf(
		"%s: public access is not blocked (all four block_public_* settings must be true)",
		[r.address],
	)
}
 
# A live exception is loud: every run says it is in force, and who owns it.
warn contains msg if {
	some e in break_glass
	live(e)
	msg := sprintf("break-glass active for %s (%s, owner %s) until %s", [e.address, e.ticket, e.owner, e.expires])
}
 
# An expired entry fails the build on its own. The hole closes itself.
deny contains msg if {
	some e in break_glass
	not live(e)
	msg := sprintf("break-glass for %s (%s) expired on %s: fix the resource or open a new, approved exception", [e.address, e.ticket, e.expires])
}
 
# This is the whole policy for the runs below. The owner-tag warn rule from
# earlier is not repeated here, so the file holds three rule bodies: two deny,
# one warn. That is the number conftest reports as tests.

terminal

conftest test tfplan.json --policy policy/
echo "exit=$?"

output

WARN - tfplan.json - main - break-glass active for aws_s3_bucket_public_access_block.marketing_site (SEC-4412, owner @dana) until 2026-08-04T00:00:00Z
 
3 tests, 2 passed, 1 warning, 0 failures, 0 exceptions
exit=0

The pipeline goes green, and the exception announces itself in the log of every single run while it is live, naming the ticket and the person who owns it. Then the fortnight passes and nobody moves the site behind a CDN (content delivery network), because that is what actually happens.

terminal

# two weeks later. Same command, same repository, nothing edited.
conftest test tfplan.json --policy policy/
echo "exit=$?"

output

FAIL - tfplan.json - main - aws_s3_bucket_public_access_block.marketing_site: public access is not blocked (all four block_public_* settings must be true)
FAIL - tfplan.json - main - break-glass for aws_s3_bucket_public_access_block.marketing_site (SEC-4412) expired on 2026-08-04T00:00:00Z: fix the resource or open a new, approved exception
 
3 tests, 1 passed, 0 warnings, 2 failures, 0 exceptions
exit=1

The expiry is the piece teams skip, and it is the whole difference between an exception and a disabled check. A `continue-on-error` added at 11pm has no owner, no reason, and no end date, and it will outlive everyone who remembers why it is there. A register entry has all three, it is loud for as long as it lives, and on the day it goes stale it fails the build by itself and puts the decision back in front of a person. Conftest ships a built-in version of the same idea, `exception` rules that suppress named `deny_*` rules and show up in the exceptions column of the summary. The register above earns its extra lines because the expiry and the owner are enforced by the policy rather than remembered by a human.

OPA policies for TerraformDrift detection

What Infrastructure as Code isInfrastructure as Code & automation What Terraform is & how it worksTerraform Why Terragrunt existsTerragrunt What OpenTofu is & why it forkedOpenTofu Python for DevSecOps cheat sheetCheat sheet DevSecOps & supply-chain security interview questionsInterview prep

Policy gates in the pipeline

The whole mechanism is one number

Give the gate something real to read

One deny, one warn, and the difference between them

Watch it block, then watch it pass

Wire it into the pipeline

Make the check required, or it is only advice

Break glass in the open

Related