OPA & Rego for compliance

Controls as declarative, tested policy.

An auditor sits across the table and asks one blunt question. Prove that every S3 bucket in production is encrypted at rest. S3 is Amazon's Simple Storage Service, the object storage where most cloud data ends up, and "encrypted at rest" means the bytes sitting on disk are scrambled. The old answer is a folder of console screenshots. Each one is stale the second you take it, and none of them says a thing about the fifty deploys that happened in between. Compliance as code answers a different way. You hand over a policy file, a test suite that proves the policy works, and a pipeline log showing the check ran on every change and blocked the one bucket that slipped through unencrypted. Open Policy Agent (OPA, a standalone program whose only job is answering "is this allowed?") and its policy language, Rego, are the engine that turns a written control into that running proof.

What OPA and Rego actually are

OPA works like a building inspector. The inspector does not own the building and never swings a hammer. You hand over the blueprint, the inspector reads it against a rule book, and back comes a verdict. That is the whole design: a small, general-purpose decision-maker, kept deliberately separate from the systems it guards. Your CI job (continuous integration, the automated build that runs on every commit), a running service, or a Kubernetes cluster hands OPA a JSON or YAML document, called the input, plus any reference data it needs. JSON (JavaScript Object Notation) and YAML are two text formats for the same kind of structured data, the sort with nested keys and lists. OPA hands back a decision. Rego is the language the rule book is written in, and it is declarative: you describe what a violation looks like and let the engine search the document for matches, rather than writing loops that walk it yourself. Because the engine does not care what the document represents, the same binary judges a Terraform plan, a Kubernetes manifest, a parsed Dockerfile or an incoming API request. Write the control once, run it at every control point. Everything here uses OPA 1.0 and later, where the modern Rego v1 keywords (if, contains, in) are on by default.

Anatomy of a Rego policy

A Rego file opens with a package declaration that gives its rules a namespace, so everything below it lives under data.<package>. The workhorse pattern for compliance is the deny set, and it behaves like the snag list an inspector leaves after walking a new build. An empty list means the place passed. Every line on it names exactly one thing that is wrong. You write it as 'deny contains msg if { ... }', which reads "add msg to the set called deny whenever this body holds true." The body is a list of conditions that must all be true at the same time. An expression like input.resource_changes[_] does the iterating for you: the underscore is a wildcard that tells OPA to try every element, which produces one deny entry per matching resource. Every element is the operative phrase, because a plan lists what you are deleting and what you are leaving untouched as well as what you are building. Each entry carries a change.actions list saying which of those it is, and a rule that ignores it will deny the bucket you are destroying, whose change.after is null, then block the deploy that was tidying it up. Helper rules such as 'encrypted(r) if { ... }' pull reusable logic into a named function you can test on its own. Comprehensions build a whole collection in a single expression, so {r.address | r := input.resource_changes[_]; not encrypted(r)} gathers the address of every offending bucket, duplicates removed. Hold on to the semantics, because they are the entire contract. Empty deny set, compliant. Non-empty deny set, non-compliant, and each message states which control was broken.

package compliance
 
# Control: every S3 bucket must declare server-side encryption of its own.
# Maps to SOC 2 CC6.1 (logical access protection). Where a bucket holds
# cardholder data it also evidences PCI DSS 4.0 requirement 3.5.1, stored
# card numbers rendered unreadable.
 
# deny is a partial SET rule: one message is added per violation found.
deny contains msg if {
    resource := input.resource_changes[_]     # iterate every planned resource
    resource.type == "aws_s3_bucket"
    resource.change.actions[_] in {"create", "update"}   # not deletes, reads or no-ops
    not encrypted(resource)                   # helper decides compliance
    msg := sprintf(
        "S3 bucket %q declares no server-side encryption of its own (violates SOC 2 CC6.1)",
        [resource.address],
    )
}
 
# every bucket name a real encryption resource covers in this plan
encrypted_bucket_names := {name |
    sse := input.resource_changes[_]
    sse.type == "aws_s3_bucket_server_side_encryption_configuration"
    count(sse.change.after.rule) > 0          # present-but-blank is not encryption
    name := sse.change.after.bucket
}
 
# helper function: true only when one of those resources names this bucket
encrypted(resource) if {
    resource.change.after.bucket in encrypted_bucket_names
}
 
# set comprehension: de-duplicated addresses of every failing bucket
unencrypted_buckets := {resource.address |
    resource := input.resource_changes[_]
    resource.type == "aws_s3_bucket"
    resource.change.actions[_] in {"create", "update"}
    not encrypted(resource)
}

Look at what each message carries. It names the control it maps to, so a failing check tells an auditor the exact requirement that broke rather than a bare red X. CC6.1 is a criterion in SOC 2 (Service Organization Control 2, the audit report customers ask software vendors for) covering who and what can reach a system, and requirement 3.5.1 of the PCI DSS (Payment Card Industry Data Security Standard) says stored card numbers must be unreadable. The second one stays in the comment rather than in the message because it only binds buckets that actually hold card data. Read the wording of the message closely as well. Since January 2023 Amazon has encrypted every new object in every bucket with its own managed key, so a bucket with nothing declared is not sitting in plaintext; what it lacks is a key you picked and can point an assessor at, and that is what this rule is really asking for. Notice too that the helper never looks inside the bucket. Since version 5 of the Terraform AWS provider there is nothing inside to look at: encryption is declared by its own aws_s3_bucket_server_side_encryption_configuration resource that points at the bucket by name. So the policy first gathers the names of every bucket one of those resources covers, then asks whether this bucket is one of them. The count check earns its keep in that gather, because a plan prints a block you never filled in as an empty list rather than leaving the key out, so an encryption resource can turn up with rule set to [], and a rule list with nothing in it is not encryption. And the file itself, kept in git and reviewed like any other code, becomes a statement of the control that a person can read and a machine can run.

Run the check: install, eval, interpret

OPA ships as one static binary with no runtime to install around it. Drop it on your path, feed it the policy plus a sample input, and ask for the deny set directly with opa eval. This is the tightest loop there is for watching a rule fire before you wire anything into a pipeline.

# Linux amd64; macOS and Windows builds are on the same downloads page.
curl -L -o opa https://openpolicyagent.org/downloads/latest/opa_linux_amd64_static
chmod 755 opa && sudo mv opa /usr/local/bin/opa
opa version

Version: 1.5.0
Build Commit: 4c6e524
Go Version: go1.24.4
Platform: linux/amd64

Now something realistic. Terraform will print the change it is about to make as JSON, using 'terraform show -json plan.out', which turns your pending infrastructure change into a document OPA can read. Watch where the encryption sits. Version 5 of the AWS provider, released in June 2023, removed the old inline server_side_encryption_configuration argument from aws_s3_bucket, so no current plan carries it. Encryption is a separate resource that names its bucket and holds the algorithm inside a rule block. Here the plan is trimmed down to two buckets and the single encryption resource covering one of them. raw_events has nothing pointing at it and should be denied. audit_logs is covered by a KMS (Key Management Service, Amazon's managed encryption keys) rule and should pass without a word.

{
  "resource_changes": [
    {
      "address": "aws_s3_bucket.raw_events",
      "type": "aws_s3_bucket",
      "change": { "actions": ["create"], "after": { "bucket": "acme-raw-events" } }
    },
    {
      "address": "aws_s3_bucket.audit_logs",
      "type": "aws_s3_bucket",
      "change": { "actions": ["create"], "after": { "bucket": "acme-audit-logs" } }
    },
    {
      "address": "aws_s3_bucket_server_side_encryption_configuration.audit_logs",
      "type": "aws_s3_bucket_server_side_encryption_configuration",
      "change": {
        "actions": ["create"],
        "after": {
          "bucket": "acme-audit-logs",
          "rule": [
            { "apply_server_side_encryption_by_default": [
                { "sse_algorithm": "aws:kms",
                  "kms_master_key_id": "arn:aws:kms:eu-west-1:111122223333:key/2f9c8b1a-3d4e-4f50-9a6b-7c8d9e0f1a2b" } ],
              "bucket_key_enabled": true }
          ]
        }
      }
    }
  ]
}

opa eval -d policy.rego -i input.json 'data.compliance.deny'

{
  "result": [
    {
      "expressions": [
        {
          "value": [
            "S3 bucket \"aws_s3_bucket.raw_events\" declares no server-side encryption of its own (violates SOC 2 CC6.1)"
          ],
          "text": "data.compliance.deny",
          "location": { "row": 1, "col": 1 }
        }
      ]
    }
  ]
}

What comes back is not a true or false. It is the deny set itself. opa eval wraps every query in a result/expressions envelope, and the value field holds the array of violation messages: one here, naming the unencrypted bucket. The audit_logs bucket had a matching encryption resource in the same plan, so it produced nothing at all. Add '--format pretty' to strip the envelope and print the messages on their own, which is what you want in front of a human reader. Keep the raw JSON for everything else: piping into jq, attaching to a ticket, or storing as machine-readable evidence with a timestamp on it.

Prove the policy is correct: opa test

A policy that never denies anything and a policy that denies everything both look green in a pipeline. One is a smoke alarm with a dead battery. The other is a smoke alarm that shrieks at toast. From across the room neither looks different from a working one, and you find out which you have on the worst possible day. Rego ships with a test framework so you prove behaviour with fixtures instead of hoping. A test is a rule whose name starts with test_. It builds a synthetic input with 'with input as', then asserts on the deny set that comes out. Ship a policy only once it has rejected a known-bad input, allowed a genuinely good one, and pinned down the half-configured case in between.

package compliance_test
 
import data.compliance
 
# a bad input MUST produce exactly one denial
test_unencrypted_bucket_denied if {
    result := compliance.deny with input as {
        "resource_changes": [{
            "address": "aws_s3_bucket.bad",
            "type": "aws_s3_bucket",
            "change": {"actions": ["create"], "after": {"bucket": "bad"}},
        }],
    }
    count(result) == 1
}
 
# a good input MUST produce no denials: bucket plus a rule that really encrypts it
test_encrypted_bucket_allowed if {
    result := compliance.deny with input as {"resource_changes": [
        {
            "address": "aws_s3_bucket.good",
            "type": "aws_s3_bucket",
            "change": {"actions": ["create"], "after": {"bucket": "good"}},
        },
        {
            "address": "aws_s3_bucket_server_side_encryption_configuration.good",
            "type": "aws_s3_bucket_server_side_encryption_configuration",
            "change": {"actions": ["create"], "after": {
                "bucket": "good",
                "rule": [{"apply_server_side_encryption_by_default": [{"sse_algorithm": "aws:kms"}]}],
            }},
        },
    ]}
    count(result) == 0
}
 
# present-but-blank is NOT encryption: empty rule list, so deny still fires
test_blank_encryption_config_denied if {
    result := compliance.deny with input as {"resource_changes": [
        {
            "address": "aws_s3_bucket.blank",
            "type": "aws_s3_bucket",
            "change": {"actions": ["create"], "after": {"bucket": "blank"}},
        },
        {
            "address": "aws_s3_bucket_server_side_encryption_configuration.blank",
            "type": "aws_s3_bucket_server_side_encryption_configuration",
            "change": {"actions": ["create"], "after": {"bucket": "blank", "rule": []}},
        },
    ]}
    count(result) == 1
}

opa test . -v

data.compliance_test.test_unencrypted_bucket_denied: PASS (312µs)
data.compliance_test.test_encrypted_bucket_allowed: PASS (204µs)
data.compliance_test.test_blank_encryption_config_denied: PASS (196µs)
--------------------------------------------------------------------------------
PASS: 3/3

Turn the verdict into an exit code

One step is left before any of this can stop a bad deploy. The deny set has to become a non-zero exit code, because the exit code is the only thing a CI runner actually reads. This is the most common mistake people make with raw opa eval, and it fails quietly, which is what makes it expensive.

# 1) opa eval alone: its exit code ignores the deny set (does NOT gate CI)
opa eval -d policy.rego -i input.json 'data.compliance.deny' >/dev/null
echo "exit=$?"
 
# 2) --fail-defined + set iteration: non-zero exit when a violation exists
opa eval --fail-defined -d policy.rego -i input.json 'data.compliance.deny[_]' >/dev/null
echo "exit=$?"

exit=0
exit=1

In production, syntax is almost never what bites you. Three other things do. False positives come first: this rule can only see the plan it was handed, so it will flag a bucket whose encryption resource lives in a different root module, or one covered by an account-wide default, even though both are encrypted. The same happens to a bucket whose name Terraform cannot settle until apply time, because the name the policy joins on comes back null in the plan. A check that cries wolf gets ignored within a week, so tune your rules against real plans and prefer explicit patterns over guesswork. Auditor acceptance is second. Assessors do accept policy as code, but only when the policy sits in version control, its test results are retained, and every message maps to a named control. An opaque true or false satisfies nobody, and a passing check is evidence toward a control, never a certificate on its own. Scale is third, and it shows up in a way peculiar to OPA. Every file you point it at is loaded into one data tree, so two hundred policies that all define deny in package compliance collapse into a single undifferentiated set of messages with no way to tell which file wrote which line. Give each control its own package, name the package after the control, and run 'opa test --coverage', which reports the lines no fixture ever reached. That report is how you find the rule that quietly stopped firing after a well-meaning edit.

Try this

Point this at a plan you did not write. In a repo at work, run terraform plan -out plan.out and then terraform show -json plan.out > input.json, and run the deny query over the result with the policy exactly as it stands here. Two things usually happen. The set names buckets nobody on the team expected, and at least one of those buckets turns out to be encrypted somewhere the plan cannot see. Before you touch the rule to deal with the second one, write the fixture for it in policy_test.rego, so the narrowing you are about to do stays as narrow as you meant it.

Takeaway

The part teams skip is the test file. A rule that names the bucket and the control is easy to admire on the afternoon you write it. policy_test.rego is what tells you, a year and forty edits later, that it still fires on a bad plan and still stays quiet on a good one. Write the failing fixture before you write the rule, the way you would for any other code you expect people to trust.

opa eval and opa test are the primitives, and every line above runs today with nothing on your machine but the OPA binary. A real pipeline wants more comfort than that: discovering every policy in a directory by itself, reading YAML and HCL (HashiCorp Configuration Language, the format Terraform files are written in) without you converting anything first, and printing a clean pass/fail summary that fails the build with no --fail-defined incantation to remember. Conftest is a thin wrapper around this same OPA engine running this same Rego, and the next lesson takes these policies into CI with it.

Related