Input, data & document model

Feeding facts to policy.

A courtroom needs two different things before anyone can rule on anything. There is the evidence for today's case, which is new every trial. And there is the law library on the shelf, which sits there unchanged until someone passes a new statute. OPA (Open Policy Agent, the engine that answers policy questions on behalf of other software) splits its world the same way. input is the evidence: the one request, manifest, or Terraform plan you are asking about right now, handed over fresh with every query.

data is the library. It holds facts you loaded ahead of time, like allowlists, role maps and exception lists, and they stay true until someone publishes new ones. Keeping the two apart is what lets a platform team exempt a new service on a Tuesday afternoon without anyone editing a rule. Facts that move on business time belong in data. Logic belongs in .rego files.

input: the document on trial

Every opa eval --input, every Conftest run, every Gatekeeper admission review takes the thing being checked and hands it to your rules as input. Inside Rego you read it like any other object: input.method for an HTTP request, input.spec.containers[_] for a Kubernetes Pod, input.resource_changes[_] for a Terraform plan exported to JSON (JavaScript Object Notation, the text format OPA works in). Conftest turns YAML and HCL (HashiCorp Configuration Language, the syntax Terraform files are written in) into that same shape. Gatekeeper puts the object being admitted at input.review.object. The structure changes with whatever you are policing. The format never does. OPA always sees JSON.

echo '{"method":"GET","path":"/api/users"}' > request.json
opa eval -d authz.rego -i request.json --format pretty 'input.path'

"/api/users"

data: the facts you load in advance

You get facts into OPA three ways: a data.json file on disk, an opa eval --data flag, or a data/ directory shipped inside a bundle. Policies then read them at paths like data.allowed_registries[_] or data.exceptions.users[_]. The habit to build is unglamorous and pays off constantly. When the allowlist gains an entry, you publish new data. You do not go hunting for deny conditions scattered across five packages.

mkdir -p policy
cat > policy/data.json <<'EOF'
{
  "allowed_registries": ["registry.internal.corp", "gcr.io/acme-prod"],
  "break_glass_users": ["oncall-admin"]
}
EOF
 
cat > policy/main.rego <<'EOF'
package main
 
deny contains msg if {
    image := input.spec.containers[_].image
    reg := split(image, "/")[0]
    not reg in data.allowed_registries
    msg := sprintf("registry %v not in allowlist", [reg])
}
EOF

# data + policy separated

echo '{"spec":{"containers":[{"name":"app","image":"docker.io/library/nginx:latest"}]}}' > pod.json
opa eval -d policy/ -i pod.json --format pretty 'data.main.deny'

["registry docker.io not in allowlist"]

One tree holds everything OPA knows

OPA keeps everything it knows in a single virtual tree rooted at data: the facts you loaded, the output of every compiled rule, some system information about itself. input sits beside that tree as its own top-level document, never inside it. After compilation your rules become addressable, so a rule called deny in package main answers at data.main.deny. That is also why opa eval data.allowed_registries returns your list the moment you load data.json. One tree, two sources feeding it, and your queries walk it the same way either way.

Swapping facts inside a test

In a test, with data.path as value replaces a fact for the length of one expression, exactly the way with input as replaces the document. Nothing on disk changes. One test file can then walk a table of cases, varying the request and the facts together, and prove the exception path behaves. No fixture directory full of near identical JSON required.

cat > policy/main_test.rego <<'EOF'
package main
 
test_break_glass_bypasses if {
    count(deny) == 0 with input as {"spec":{"containers":[{"name":"a","image":"docker.io/x"}]}}
        with data.break_glass_users as ["alice"]
        with data.allowed_registries as ["registry.internal.corp"]
}
EOF
 
opa test policy/ -v

data.main.test_break_glass_bypasses: PASS
PASS: 1/1

Bundles carry facts and rules together

A bundle is a tarball, and it holds both halves. The .rego files and the data.json tree travel together, get pulled together, and switch over together (the bundles lesson covers the mechanics). That is how a sidecar or an admission controller picks up a new allowlist with no application redeploy at all. Sign your bundles. Unsigned, anyone who can write to the storage behind them can widen your exceptions and nothing downstream will blink.

opa build -b policy/ -o /tmp/bundle.tar.gz
tar tzf /tmp/bundle.tar.gz | head

/.manifest
/data.json
/main.rego
/main_test.rego

Whoever edits data is writing policy

Policies decide using data, so anyone who can edit data can change what your policy does without touching a rule. One username added to break_glass_users. One more hostname slipped into allowed_registries. No Rego diff, same outcome. Give data files the review, the versioning and the access control you give .rego files, and never let the workloads being policed write their own exemption list.

A list of forty registries is fine as a plain array. A list of forty thousand images is not, if every rule walks it from the top on every request. Shape large allowlists as objects keyed by the value you look up, so data.allowed_images[img] is one lookup rather than img == data.list[_] scanning the lot in a hot path. Generate that keyed JSON in a pipeline from whatever the source of truth already is, a CSV export or an IAM (identity and access management) dump, instead of maintaining it by hand.

Patching a live OPA

OPA's HTTP API lets you write documents straight into the tree with a PUT to /v1/data, which is genuinely useful at 3am when the on-call engineer needs an exception faster than a bundle build. It is also a hole in your audit trail. A new signed bundle is the answer you want almost every time. If you keep the PUT path open, put it behind RBAC (role-based access control, rules about which identity may call which endpoint) and log every single call.

curl -s -X PUT localhost:8181/v1/data/config -d '{"break_glass": true}'

{}

Version data separately from code when the two change at different speeds. allowed_registries might move daily while your deny logic holds steady for months. Some teams run two bundles for that reason: the policy bundle rebuilt weekly, the data bundle rebuilt on every allowlist change, with OPA configured to merge them at non-overlapping roots. Watch the .manifest roots when you do. If two bundles claim the same root, OPA refuses the update and quietly carries on serving the old policy.

Pulling facts over the network

OPA can also reach out over HTTP and pull JSON into data while it runs, or have facts pushed in from an external service. Keep that rare. Every remote call added at evaluation time is one more thing that can be slow, unreachable, or wrong at the exact moment a decision is due. Bundles stay the default because they are versioned, signed, and cached on the OPA side.

Write down every data path your policies read, in a schema README sitting next to the policy. The day someone renames data.allowed_registries to data.v2.registries, that list is what you grep and those are the tests that fail loudly. Undocumented paths are coupling you cannot see until enforcement stops without a sound.

Bundle storage deserves the same care as any other artifact store. Turn on encryption at rest (S3 SSE or GCS CMEK, the server-side encryption options in Amazon and Google object storage) and cut IAM down to two roles: the CI job that publishes, and the OPA instances that read. A threat model that only asks what happens if someone merges bad Rego is missing the other half. Publishing a bad bundle gets you the same place.

Running policy for many customers on one OPA? Shard the facts by tenant. data.tenants[tid].allowlists, with tid taken from input and used as the index, means a lookup can only ever reach one customer's list. A single flat allowlist shared across tenants is a cross-tenant leak waiting for the first sloppy rule.

Keeping data.json in Git works nicely while the allowlist is something a person could read in one sitting. Past a few thousand rows it stops working. Generate the file in CI (continuous integration, the pipeline that runs on every commit) from a warehouse export and publish it as a bundle artifact instead. Git is version control, not a database.

Point the same CODEOWNERS entries at your data files as at your Rego. A pull request that adds one line of JSON can widen allowed_registries further than any rule change sitting next to it in the same diff, and it will look smaller to a tired reviewer at 5pm.

If facts come out of Vault or AWS Parameter Store on the way into a bundle, print the lineage in the CI log: which secret version produced the data.json that ended up in bundle digest abc123. Nobody wants that trail until the week they badly need it, and by then it is too late to start collecting.

Validate the shape of data.json in CI before the bundle is built, with a JSON Schema check or a custom regal rule. A key typed as allowed_registry instead of allowed_registries raises no error anywhere. The rule reads an undefined path, the deny never fires, the pipeline goes green, and enforcement is off.

If the facts are exported from a live database, decide how stale they are allowed to get and write that number down. Role data ten minutes old means an employee offboarded nine minutes ago still authorizes. That might be acceptable. It might not. Either way it should be a number you chose, trading freshness against the load of polling more often.

Wire break-glass lists to the offboarding webhook from your HR (human resources) system so removals happen without a human. Hand-edited JSON misses terminations. It always does, eventually. A pipeline with an audit log beats a quarterly spreadsheet review nobody wants to run.

What to ask on a data pull request

Treat a data change like a database migration, because that is roughly what it is. Every data pull request should answer five things: who signed off, which environments will consume it, when it stops being true if it is temporary, how large the diff really is, and whether opa test still passes with the new facts loaded.

If the file carries anything sensitive, internal team names or customer identifiers, encrypt it in Git with SOPS (Secrets OPerationS, a tool that encrypts the values inside a YAML or JSON file while leaving the keys readable). Policy repos get cloned widely inside a company. More people can read yours than you think.

Exception entries need one field the others do not: an end date. Record who asked for the entry, which ticket or exception process approved it, which environments it covers, and the day it expires. Allowlist entries with no expiry pile up quietly. Enough of them and the policy still runs beautifully while applying to almost nobody.

One test belongs in every policy repo: override the facts with with data.roles as {} and assert the rule still denies. That is the regression that catches a fail-open data patch, where somebody renames a key or ships an empty file and every request starts sailing through because the lookup returns nothing at all.

Try this

Pull the allowlist out of the rule body and into JSON, then flip who is on it. alice is an admin, eve is not, and one unchanged rule handles both. Notice how this policy differs in shape from the earlier one. policy/main.rego builds a deny set, a collection of messages where empty means nothing was wrong, while authz.rego here is a single allow boolean that starts false and only turns true when the rule matches. Both read data the same way. When tomorrow's exception lands you edit data/roles.json, the rule body stays put, and the tests pin both answers.

mkdir -p data
echo '{"admins":["alice","bob"]}' > data/roles.json
cat > authz.rego <<'EOF'
package authz
default allow := false
allow if input.user in data.roles.admins
EOF
opa eval -d authz.rego -d data/ -i <(echo '{"user":"alice"}') --format pretty 'data.authz.allow'
opa eval -d authz.rego -d data/ -i <(echo '{"user":"eve"}') --format pretty 'data.authz.allow'
opa test . -v

true
false

Takeaway

input is the thing being judged this second. data is what OPA already believed before the question arrived. Both feed the same rules, and usually only one of them gets reviewed.

So make the boring move: ship data inside signed bundles in production, and put the allowlist pull request in front of the same reviewers, with the same seriousness, as the rule that reads it.

Related