Drift as a compliance signal

Out-of-band change = potential control failure.

Your SOC 2 auditor (SOC 2 is the report an outside firm writes to say you handle customer data the way you claim you do) asks for evidence that production restricts SSH to the bastion network. SSH is the remote-login protocol engineers use to get a command prompt on a server, and the bastion is the single hardened machine everyone has to hop through first. You show the Terraform that declares security group app-ssh with ingress on port 22 limited to 10.0.1.0/24, plus the deploy-time scan that passed. Then the auditor asks the sharper question: how do you know it is still configured that way today? Three weeks ago an on-call engineer widened that group in the console at 2 a.m. to debug an outage and never reverted it. Live state now allows 0.0.0.0/0 on port 22, meaning every address on the internet, the exact thing CIS AWS Foundations control 5.2 forbids (CIS is the Center for Internet Security, whose benchmarks are the standard hardening checklists for cloud accounts). Your reviewed code still says 10.0.1.0/24. The control you certified is no longer operating, and nothing in your deploy pipeline would catch it, because the deploy already happened. That gap is drift.

What drift is, and why an auditor cares

A landlord files a floor plan with the city, and the building is supposed to match it. One weekend somebody knocks a doorway through a wall and never updates the plan. The building still stands. The paperwork is now a lie. Configuration drift is that, for cloud resources: any divergence between the live state of a resource and the approved, version-controlled baseline that describes it, which is your Terraform, CloudFormation, or Pulumi code. An out-of-band change is any modification made outside that pipeline, whether it is a console click, a stray CLI call (command-line interface, the terminal tool that talks to the cloud provider), an auto-remediation bot, or another team's script. Infrastructure as code tools, IaC for short, split the world in two. Managed resources are present in state and owned by your code. Unmanaged resources are live but were never described in code at all. Drift happens to managed resources; unmanaged resources are a related blind spot you will meet in a minute. The baseline is the artifact you reviewed and certified, so anything that moves live state away from it moves you away from the posture you attested to.

In a compliance program, unexplained drift on a security-relevant resource is a potential control failure rather than a cosmetic difference. A security group opened to the internet. Encryption toggled off. A public-access block removed. Flow logs disabled. Each one is a specific control regressing. The drifted app-ssh group maps straight onto CIS AWS Foundations 5.2, onto the inbound-restriction requirements in PCI DSS (Payment Card Industry Data Security Standard, the rules that apply once you touch cardholder data), and onto SOC 2 CC6.1, the criterion about protecting your network boundary. Maybe it was an honest mistake that weakened the control. Maybe it was an attacker quietly loosening it. Either way, a control you told an auditor was operating may not be, and catching that is the whole job of continuous compliance.

resource "aws_security_group" "app_ssh" {
  name        = "app-ssh"
  description = "SSH to the app tier from the bastion subnet only"
  vpc_id      = aws_vpc.main.id
 
  ingress {
    description = "SSH from bastion subnet"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["10.0.1.0/24"]   # bastion only -> CIS AWS 5.2 compliant
  }
 
  tags = {
    Control = "CIS-AWS-5.2"
    Owner   = "platform"
  }
}

Catch drift with terraform plan -detailed-exitcode

A stocktake finds missing inventory by walking the shelves and comparing what is actually there against what the ledger claims. Terraform does the same thing when it refreshes. It reads the real configuration of every managed resource from the cloud API (the programmatic interface a provider exposes so software can query and change things) and diffs it against the last-applied state. Plain terraform plan mixes two different stories into that diff. One is drift, live versus state. The other is pending config changes, state versus the code you edited but never applied. For a pure drift check you want -refresh-only, added in Terraform 0.15.4, which reports only what changed outside Terraform and proposes no config-driven actions. Pair it with -detailed-exitcode, which makes the exit code machine-readable. The exit code is the small number a program hands back to the shell when it finishes: 0 means the plan is empty and nothing drifted, 1 means the command itself errored, 2 means a diff exists. Exit 2 is your drift signal, the one line a scheduled job needs to alarm on.

terraform init -input=false
terraform plan -refresh-only -detailed-exitcode -no-color -out=drift.plan

aws_security_group.app_ssh: Refreshing state... [id=sg-0f3a9c21b7de4a1c8]
 
Note: Objects have changed outside of Terraform
 
Terraform detected the following changes made outside of Terraform since the
last "terraform apply" which may have affected this plan:
 
  # aws_security_group.app_ssh has changed
  ~ resource "aws_security_group" "app_ssh" {
        id = "sg-0f3a9c21b7de4a1c8"
      ~ ingress = [
          + {
              + cidr_blocks = ["0.0.0.0/0"]
              + from_port   = 22
              + to_port     = 22
              + protocol    = "tcp"
            },
            # (1 unchanged element hidden)
        ]
        # (7 unchanged attributes hidden)
    }
 
This is a refresh-only plan, so Terraform will not take any actions to undo
these changes.
 
$ echo $?
2

The refresh-only plan names exactly one thing that changed outside Terraform: app-ssh gained an ingress rule allowing 0.0.0.0/0 on port 22. That is CIS 5.2 failing in production while the code and the deploy-time scan are both green. echo $? prints the exit code of the last command, and it returns 2, so nobody has to read the diff to know something moved. You wire that exit code into a scheduled pipeline that fails the job, pages on-call, and opens a ticket. One detail in the gate script below is deliberate: there is no set -e. With errexit switched on, the shell would abort the moment plan exits non-zero, and you would never reach the line that reads the code.

#!/usr/bin/env bash
set -uo pipefail   # note: NOT -e, so we can read plan's exit code ourselves
 
terraform plan -refresh-only -detailed-exitcode -no-color -out=drift.plan
code=$?
 
case "$code" in
  0) echo "No drift - live state matches the certified baseline." ;;
  2) echo "::error::Drift detected - CIS AWS 5.2 may be violated on app-ssh"
     terraform show -no-color drift.plan | tee drift-evidence.txt
     exit 1 ;;                 # fail the job -> alert + ticket, do NOT auto-apply
  *) echo "terraform plan failed (exit $code)"; exit "$code" ;;
esac

driftctl: managed drift and the resources nobody owns

A stocktake only counts what is already on the ledger. If a pallet was never written down in the first place, walking the shelves will not flag it. terraform plan has the same structural blind spot: it can only evaluate resources already in its state. A security group someone created entirely in the console, never described in any Terraform, is invisible to it. driftctl scan closes that gap. It compares your state files against live infrastructure and reports coverage, managed resources that have drifted, and its real edge, unmanaged resources that no code owns. The --error-on-changes flag turns any drift or unmanaged finding into a non-zero exit (1) so a pipeline can gate on it. One honest caveat for 2025/2026: driftctl is no longer actively developed. Snyk moved it to maintenance mode and its final release, v0.40.0, shipped in December 2023. Plenty of teams now get the unmanaged-resource view from a CSPM (cloud security posture management tool, which scans the account itself rather than your state files) or from AWS Config instead. The exit-code gating pattern is identical whichever tool emits it.

driftctl scan \
  --from tfstate+s3://tf-state-prod/network/terraform.tfstate \
  --error-on-changes

Scanning resources:
Found 46 resource(s)
 - 87% coverage
 - 40 resource(s) managed by Terraform
     - 1 resource(s) out of sync with Terraform state (drifted):
         aws_security_group.app_ssh (sg-0f3a9c21b7de4a1c8)
           ~ ingress: added 0.0.0.0/0 on tcp/22
 - 6 resource(s) not managed by Terraform (unmanaged):
     aws_security_group (sg-0d91b8ac2f5e07d43)  ->  0.0.0.0/0 on tcp/3389
 
$ echo $?
1

Investigate first, reconcile second, prevent where you can

When drift fires, investigate before you touch anything. A blind terraform apply is the classic trap. The change you are about to revert might be a legitimate emergency fix that is holding the service up, and the reconciling plan might show a destroy-and-recreate that causes a fresh outage tonight. Once you understand it, reconcile. If the change was intentional and compliant, codify it: backport it into the code, open a PR (pull request, the review step before code merges), get it reviewed, re-apply. If it weakened a control, revert it: restore the baseline, then treat it as a possible security incident. Better than detecting drift is preventing it. Service control policies (SCPs, the account-wide guardrails in AWS Organizations), Google Cloud Org Policy, Kubernetes admission controllers, and an IaC-only change workflow block console edits outright, so the drift never happens. Detection catches what prevention misses, and a mature program runs both.

At scale the failure mode that kills these programs is false positives. Provider-side normalization produces phantom drift: IAM policy documents (identity and access management, the rules for who is allowed to do what) that come back with their JSON keys reordered, security-group rules shuffled into a different order, an autoscaling desired_count that legitimately floats up and down with load. An alarm that cries wolf every morning teaches people to close the ticket without reading it. Scope the check to security-relevant resources, use lifecycle { ignore_changes } for the attributes that genuinely move on their own, and run it per workspace on a schedule so the refresh API calls do not hit rate limits across hundreds of state files. For the auditor, the drift job itself is the evidence. Its schedule, its logs, the captured terraform show output, the exit code, and the linked remediation ticket together demonstrate the kind of continuous monitoring SOC 2 CC7 asks about: proof that the control is watched between deploys, not only on the day it ships.

Drift detection is the after-the-fact half of infrastructure compliance, the half that catches controls quietly regressing weeks after they shipped. The other half is stopping non-compliant infrastructure from shipping in the first place, by running the same framework-mapped policies as a build gate. That is the next lesson, IaC compliance gates, where the exit code you now page on becomes the exit code that fails a pull request.

Try this

Work through “Investigate first, reconcile second, prevent where you can” yourself on a sandbox you can throw away, following the commands above in order. Then break one step deliberately and re-run, so you have seen the failure before it finds you.

Takeaway

The trap worth remembering here: a blind apply to 'fix' drift can cause tonight's outage. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related