>_SecOpsLog_
CoursesCompliance as codeCompliance at scale

Compliance at scale

Inherited guardrails, secure-by-default accounts.

Expert30 min · lesson 13 of 15

A supermarket chain with sixty stores does not check freezer temperatures by driving to each store. Every freezer reports its own temperature to one screen at head office, and a manager reads sixty numbers in a second. Compliance at scale works the same way. An auditor opens your SOC 2 renewal (Service Organization Control 2, the audited report customers ask for before they trust you with their data) with a request that sounds small: prove storage encryption is enforced on every database and every bucket, across all sixty of your AWS accounts (Amazon Web Services accounts, each one its own billing and security boundary), for the whole audit period. You cannot log into sixty accounts by hand. Green screenshots from three of them say nothing about the other fifty-seven. The answer is a single query, and that query only works because the control behind it was written once, shipped everywhere, evaluated in every account, and rolled back up to one place you can read in seconds.

Two halves: locks and cameras

Compliance work splits in two, like locks and cameras on a building. The preventive half is the lock. A policy engine running in CI (continuous integration, the automated build that fires on every change) or at admission refuses a bad Terraform plan (the preview file showing what a change is about to create or alter) or a bad Kubernetes manifest before it can merge. The detective half is the camera. It keeps watching what actually exists and reports the drift the lock never saw: a resource somebody clicked into being in the console, an older resource that predates the policy, a control the pipeline gate cannot check by reading a plan file. Both halves have the same scaling problem and the same fix. Write each control in exactly one place, then make every account and every pipeline read that one definition instead of a copy that quietly wanders off. The rest of this lesson is that idea done twice, once for detection and once for prevention.

Detective half: roll every account into one query

AWS Config is a tape recorder for infrastructure. It writes down each resource's configuration as it changes and keeps re-checking it against Config rules, which are named checks (managed ones AWS ships, or custom ones you write) such as s3-bucket-server-side-encryption-enabled. Every resource comes out stamped COMPLIANT or NON_COMPLIANT. On its own, that recorder is deaf outside its own account and its own region, which gets you nowhere with sixty accounts. An aggregator is what makes it estate-wide: a read-only viewing window that collects evaluation results from every account and every region in your AWS Organization (the tree that owns all your accounts under one root) into a single place. You create it once, in a delegated administrator account (the one account the organization nominates to run this on everyone's behalf), using an organization aggregation source. Newly vended accounts, the ones your account factory stamps out later, then show up on their own and you never touch the aggregator again. Nothing here blocks anything. An aggregator watches and totals, which is exactly the population-level evidence an auditor wants to see.

create an organization-wide Config aggregator (delegated-admin account)
# One aggregator that sees ALL org accounts + all regions, forever
aws configservice put-configuration-aggregator \
  --configuration-aggregator-name org-compliance \
  --organization-aggregation-source \
      RoleArn=arn:aws:iam::210987654321:role/AWSConfigAggregatorRole,AllAwsRegions=true
output: the aggregator now spans the whole organization
{
    "ConfigurationAggregator": {
        "ConfigurationAggregatorName": "org-compliance",
        "OrganizationAggregationSource": {
            "RoleArn": "arn:aws:iam::210987654321:role/AWSConfigAggregatorRole",
            "AllAwsRegions": true
        },
        "CreationTime": "2026-07-14T08:02:11.503000+00:00"
    }
}

Aggregation is eventually consistent. Results trickle in over minutes rather than landing the instant you press enter, so give it time to collect before you read anything into a thin answer. Then ask the rollup question. describe-aggregate-compliance-by-config-rules returns one row per rule, per account, per region, with a contributor count of how many resources are failing. That is your scoreboard for the whole estate: one command, every account.

roll up every NON_COMPLIANT rule across all accounts
aws configservice describe-aggregate-compliance-by-config-rules \
  --configuration-aggregator-name org-compliance \
  --filters ComplianceType=NON_COMPLIANT \
  --query 'AggregateComplianceByConfigRules[].[ConfigRuleName,AccountId,AwsRegion,Compliance.ComplianceContributorCount.CappedCount]' \
  --output table
output: three accounts, four failing controls, at a glance
---------------------------------------------------------------------------------
|                    DescribeAggregateComplianceByConfigRules                    |
+---------------------------------------------+---------------+-----------+------+
|  rds-storage-encrypted                      |  111122223333 |  us-east-1|  3   |
|  s3-bucket-server-side-encryption-enabled   |  111122223333 |  us-east-1|  4   |
|  s3-bucket-server-side-encryption-enabled   |  444455556666 |  eu-west-1|  1   |
|  cloudtrail-enabled                         |  777788889999 |  us-east-1|  1   |
+---------------------------------------------+---------------+-----------+------+

The scoreboard names the failing account. get-aggregate-compliance-details-by-config-rule names the failing resources, so the ticket you file lists four buckets by name instead of saying 'something in prod is unencrypted'. Notice that this detail call takes one account and one region, and will not take more. That limit is on purpose. It stops a single command from trying to hand back millions of results across the whole organization, and it makes you walk the accounts deliberately, one at a time.

drill into the one failing account for exact resource IDs
aws configservice get-aggregate-compliance-details-by-config-rule \
  --configuration-aggregator-name org-compliance \
  --config-rule-name s3-bucket-server-side-encryption-enabled \
  --account-id 111122223333 --aws-region us-east-1 \
  --compliance-type NON_COMPLIANT \
  --query 'AggregateEvaluationResults[].EvaluationResultIdentifier.EvaluationResultQualifier.ResourceId'
output: the exact buckets to remediate, named
[
    "acme-logs-staging",
    "acme-analytics-scratch",
    "acme-tmp-exports",
    "acme-user-uploads-dev"
]

A rollup is only ever as complete as the rules and recorders running underneath it. The way to guarantee that coverage without touching accounts by hand is a conformance pack, a versioned collection of Config rules deployed through the organization so every account you have today and every account you create next month runs the same set. Deploy the recorder and the pack organization-wide first. The aggregator then has something real to total in each account. Get that order wrong and the rollup will cheerfully report 100% compliant for accounts that never ran the rule at all. Auditors do accept an aggregator report as evidence, because it is machine-generated, timestamped, and covers the whole population instead of a hand-picked sample. They will also ask what that population was, so keep the account list and the conformance-pack deployment status ready to sit next to the percentage. On its own, the percentage is evidence toward the control and nothing more. The auditor still writes the narrative and still chooses how to sample.

Preventive half: ship one versioned policy everywhere

Detection tells you what already broke. To stop it happening again you put a gate in every pipeline, ahead of the merge. Conftest runs Rego policies (Rego is the rule language of Open Policy Agent, or OPA, a general-purpose engine for writing rules about structured data) against config that has structure: Terraform plans, Kubernetes manifests, Dockerfiles. When a control fails, it exits non-zero. The scale trap here is copy-paste. A hundred repositories each carrying their own copy of the encryption policy is a hundred policies, and they start drifting apart the day someone fixes one of them. The fix is a policy bundle. Write the controls once in a central repository, build a versioned artifact from that directory, publish it to an OCI registry (Open Container Initiative, the same registry format your container images already use), and have every pipeline pull a pinned version. The control is now byte-identical in all hundred repositories, and rolling out a new control across the estate is a one-line version bump instead of a hundred pull requests.

policy/rds_encryption.rego: one control, authored centrally
package main
 
# SOC 2 CC6.1 / CIS AWS 2.3.1 — RDS instances must encrypt storage at rest
deny contains msg if {
	some rc in input.resource_changes
	rc.type == "aws_db_instance"
	not rc.change.after.storage_encrypted
	msg := sprintf("RDS '%s': storage_encrypted must be true (CIS 2.3.1)", [rc.address])
}

Build the policy directory into a versioned artifact and publish it once. The version tag on the OCI reference is the pin every pipeline consumes. If you also run an OPA bundle server, opa build --revision stamps that same version string into every decision log, so any single allow or deny can be traced back to the exact policy that produced it. That provenance is what auditors keep asking for. Build once, push once, and v3.2.0 becomes the one copy that matters.

build a versioned bundle and publish it to the registry
# For an OPA bundle server: stamp the revision into every decision log
opa build -b policy/ -o bundle.tar.gz -r v3.2.0
 
# For CI distribution: publish the policy dir to an OCI registry the org can read
conftest push ghcr.io/acme/compliance-bundle:v3.2.0
output: v3.2.0 is now the single source of truth
$ ls -lh bundle.tar.gz
-rw-r--r--  1 ci ci  4.1K Jul 14 08:20 bundle.tar.gz
 
$ conftest push ghcr.io/acme/compliance-bundle:v3.2.0
2026/07/14 08:20:04 pushing bundle to: ghcr.io/acme/compliance-bundle:v3.2.0
2026/07/14 08:20:04 pushed bundle with digest: sha256:9f4c1e42...b7a2

Every pipeline's compliance step is now the same three lines, pinned to a version. The repository holds a tag, not a copy of the policy. conftest pull downloads the pinned bundle into ./policy, and conftest test runs it against the plan. Pass and it exits 0, so the stage goes green. Fail and it exits 1, and because CI treats any non-zero exit as a failed stage, the merge is blocked. That exit code is the entire enforcement mechanism. Drop it, or soften the run with --no-fail, and what you are left with is a linter nobody reads.

.gitlab-ci.yml / any pipeline: pull the pinned bundle, then gate
terraform show -json tfplan > plan.json
conftest pull oci://ghcr.io/acme/compliance-bundle:v3.2.0   # downloads to ./policy
conftest test plan.json
echo "exit=$?"
output: the gate blocks the merge, exit 1 fails the stage
FAIL - plan.json - main - RDS 'aws_db_instance.analytics': storage_encrypted must be true (CIS 2.3.1)
 
4 tests, 3 passed, 0 warnings, 1 failure, 0 exceptions
exit=1

Central distribution cuts both ways. Pinning versions is what keeps the estate consistent, and it is also what saves you the morning a stricter v3.3.0 that every pipeline auto-pulls turns a hundred merge requests red before lunch. One central policy means one central blast radius. Treat the bundle like any other production artifact. Promote a new version through a canary set of repositories first. Write approved, time-boxed deviations into the central policy as explicit exceptions, rather than letting teams scatter per-repo skips nobody can audit later. Let the slow teams sit on v3.2.0 until they upgrade on purpose. Pinning is the seatbelt that lets a shared policy keep evolving without taking the whole estate down in one synchronized bang.

A green rollup can hide accounts nobody ever checked
An aggregator only totals evaluations that exist. An account with no Config recorder, or a region where the rule was never deployed, contributes zero results. It never appears in the NON_COMPLIANT rollup, and absent reads exactly like clean. The honest answer for that account is unknown, not compliant. After every wave of account vending, reconcile the aggregator's source accounts and your conformance-pack coverage against the real organization account list before you trust a 100% figure. Deploy recorders and rules through the organization rather than account by account, so coverage grows with the estate instead of quietly falling behind it.
Author once, distribute everywhere, roll up to one query
Author once, centrally
Org-root guardrails
SCPs / conformance pack
Versioned policy bundle
opa build -r v3.2.0 -> OCI registry
Distribute to the whole estate
Config rules org-deployed
every account + region, auto-enrolled
Pipelines pull a pinned version
conftest pull ...:v3.2.0
Roll results back up
Org Config aggregator
one query, all accounts
Compliance % per rule/account
population-level auditor evidence
One control, written once, pushed to every account and every pipeline, then totalled by a single query. Scale comes from a central source plus a rollup, never from per-account effort.
Quick check
01Your org aggregator reports s3-bucket-server-side-encryption-enabled at 100% COMPLIANT. You also vended 12 new accounts last week and nobody has touched their config. What is the most likely explanation?
Correct — An aggregator only totals evaluations that exist, and a missing recorder or missing rule produces no data. No data looks identical to a pass. Deploy the recorder and the conformance pack organization-wide, then reconcile coverage against the real account list.
Incorrect — No. The organization aggregation source does pick up new accounts for reporting, but it never deploys a recorder or a rule, and without those there is nothing in the account to evaluate.
Incorrect — No. A Service Control Policy is preventive and enforced somewhere else entirely. A Config result is a separate detective evaluation, and producing one still needs a recorder and a rule in that account.
Incorrect — No. CappedCount limits the contributor count shown on a row. It has nothing to do with which accounts are covered, and it never turns real failures into a clean report.
02A team is tired of red pipelines, so they change their compliance stage to run conftest with --no-fail. The bundle is still pulled at the pinned v3.2.0. What have they actually changed?
Incorrect — No. Pinning decides which policy runs, not what happens when it finds a violation. Enforcement comes from the non-zero exit code, and that is exactly what the soft run removes.
Incorrect — No. Conftest evaluates a plan file in the pipeline and sends nothing to AWS Config. The aggregator only ever totals Config rule evaluations of resources that already exist.
Correct — A non-zero exit is the whole mechanism that turns an advisory policy into an enforced gate. Take it away and the control still runs and still prints, but nothing is blocked, which leaves you with a linter nobody reads.
Incorrect — No. Nothing in a pipeline rebuilds or republishes the bundle. The central policy at v3.2.0 is untouched, and only this one repository's gate has gone soft.
03The rollup table shows s3-bucket-server-side-encryption-enabled failing in account 111122223333, region us-east-1, with a contributor count of 4. Your remediation ticket needs the bucket names. What do you run next?
Incorrect — No. That command returns one row per rule, account and region with a count, and never resource IDs. Dropping the filter only adds the passing rules to the table.
Correct — The scoreboard names the account, and the detail call names the resources, which is how the ticket ends up listing four buckets by name instead of saying something in prod is unencrypted.
Incorrect — No. That call is deliberately scoped to one account and one region. The constraint is what stops a single query from trying to return millions of results across the estate, so you page through accounts on purpose.
Incorrect — No. The gate reads a plan file, which describes changes about to be made. These buckets already exist, and some of them may never appear in any plan. Naming existing resources is the detective side's job.

What you eventually hand the auditor is three artifacts that have to agree with each other: the aggregator query and its result, the bundle version that produced every pipeline decision, and the account list the query actually ran over. That package is evidence toward the control, not a passed audit, because the auditor still writes the narrative and still picks the sample. Assembling it deliberately, before anyone asks, is where the next lesson on audit readiness begins.

Try this

Work through “Preventive half: ship one versioned policy everywhere” yourself on a sandbox you can throw away, following the commands above in order. Then break one step deliberately and re-run, so you have seen the failure before it finds you.

Takeaway

The trap worth remembering here: a green rollup can hide accounts nobody ever checked. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Remediation & preventionAudit readiness

Related

What OPA is: decoupled policyOPA & RegoOPA architecture and the Rego mental modelPolicy-as-code at scaleWhat IaC scanning findsCheckov & IaC scanningPolicy-as-code interview questionsInterview prep