Policy, scanning & least privilege

cfn-guard, cfn-nag, stack policies.

A CloudFormation template is a blueprint for a building, written as a text file that AWS CloudFormation (Amazon's service for turning that file into real cloud resources) reads and builds from. Before anyone pours concrete, a careful builder wants three things: an inspector who red-pens the drawings, a locked gate so only the licensed contractor swings a hammer, and a rule that nobody knocks down the load-bearing wall during the remodel. Cloud security here is those same three controls. Scan the template while it is still text. Deploy it through a narrow, purpose-built role instead of your own admin login. Pin down your irreplaceable resources so a routine update cannot quietly rebuild your production database.

Scan the template before it becomes real

Static analysis reads the template as words on a page and flags dangerous intent before a single resource exists. That is the cheapest place there is to catch a problem, because nothing has been built, nothing is costing money, and nothing is exposed to the internet yet. Two tools own this job, and they take opposite approaches.

cfn-nag (a scanner written in Ruby whose name is a play on nagging you about bad config) ships with an opinionated built-in rulebook. You point it at a file and it complains about the usual suspects: firewalls open to the whole internet, unencrypted disks, wildcard permissions, public storage buckets. You author no rules of your own.

Resources:
  AppSecurityGroup:
    Type: AWS::EC2::SecurityGroup      # a virtual firewall around a server
    Properties:
      GroupDescription: App server access
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 22                 # port 22 is SSH, the remote-login port
          ToPort: 22
          CidrIp: 0.0.0.0/0            # 0.0.0.0/0 means the entire internet

That CidrIp of 0.0.0.0/0 (CIDR, Classless Inter-Domain Routing, is the a.b.c.d/n way of writing a range of addresses, and /0 covers every address there is) opens SSH (Secure Shell, the protocol you use to log into a server's command line) to the whole world. Bots scan the internet for exactly this, around the clock. Here is what the scanner makes of it.

gem install cfn-nag
cfn_nag_scan --input-path template.yaml
echo "exit code: $?"

Successfully installed cfn-nag-0.8.10
 
------------------------------------------------------------
template.yaml
------------------------------------------------------------------------------------------------------------------------
| FAIL F1000
|
| Resources: ["AppSecurityGroup"]
| Line Numbers: [2]
|
| Missing egress rule means all traffic is allowed outbound. Make this explicit if it is desired configuration
------------------------------------------------------------
| WARN W2
|
| Resources: ["AppSecurityGroup"]
| Line Numbers: [2]
|
| Security Groups found with cidr open to world on ingress. This should never be true on instance. Permissible on ELB
------------------------------------------------------------
| WARN W9
|
| Resources: ["AppSecurityGroup"]
| Line Numbers: [2]
|
| Security Groups found with ingress cidr that is not /32
 
Failures count: 1
Warnings count: 2
exit code: 1

The exit code is 1 because there was exactly one failure, and cfn-nag returns the number of failures as its exit status, so any non-zero value fails a CI (continuous integration, the automated checks that run on every proposed change) step. Read the findings, though. The genuinely alarming one, SSH open to the entire internet, came back as W2, a warning, not a failure. Warnings do not add to the exit code by default, so on their own they would not have stopped anything.

Here is the trap. By default only failures (the F-codes) add to cfn-nag's exit status; warnings (the W-codes) leave it at zero. Plenty of real risk, including a security group open to 0.0.0.0/0, is filed as a warning because it is defensible on a load balancer. So run cfn_nag_scan --fail-on-warnings and every violation counts toward the exit code, and you triage the noise from there. A scanner whose output you always ignore is not a control.

cfn_nag_scan --input-path template.yaml --fail-on-warnings
echo "exit code: $?"

Failures count: 1
Warnings count: 2
exit code: 3

Same three findings, but the exit code is now 3, the total number of violations, so the world-open SSH rule blocks the merge on the pull request (a proposed change waiting for review) the way it should. Notice the printed counts did not move. The flag changes what feeds the exit code, not how findings are labelled.

Write your own guardrails with cfn-guard

cfn-nag knows generic best practice. It does not know that your company requires every storage bucket to block public access and every database to be encrypted, no exceptions. That is the gap cfn-guard (AWS's own policy-as-code engine) fills. You write the rules in a small DSL (domain-specific language, a tiny language built for one job), keep them in a file you review and version like any other code, and run the same file in every repository. A rule stops being a line in a wiki that a reviewer might remember, and becomes a check that fails identically everywhere.

The language is small. Bind a group of resources with let, then write a rule that asserts something about their Properties. Gate the rule with a when clause so it fires only when the relevant resources actually exist, otherwise a template with no buckets would fail a bucket rule for nothing. cfn-guard ships as a single binary: install it with the official installer script, grab a prebuilt binary from its GitHub releases page, or run cargo install cfn-guard if you already have the Rust toolchain (Rust's compiler and its cargo package manager).

# Bind every S3 bucket declared in the template
let s3_buckets = Resources.*[ Type == 'AWS::S3::Bucket' ]
 
# Fire only when the template declares at least one bucket
rule s3_public_access_blocked when %s3_buckets !empty {
  %s3_buckets {
    Properties.PublicAccessBlockConfiguration {
      BlockPublicAcls       == true
      BlockPublicPolicy     == true
      IgnorePublicAcls      == true
      RestrictPublicBuckets == true
    }
    <<
      Violation: every S3 bucket must block all four public-access settings.
    >>
  }
}
 
# Require encryption at rest on every RDS database instance
rule rds_encrypted when Resources.*[ Type == 'AWS::RDS::DBInstance' ] !empty {
  Resources.*[ Type == 'AWS::RDS::DBInstance' ] {
    Properties.StorageEncrypted == true
  }
}

S3 is Simple Storage Service (AWS's file buckets) and RDS is Relational Database Service (its managed databases). The four PublicAccessBlockConfiguration settings are the four separate switches that together keep a bucket private, and the rule demands all four. The angle-bracket block is a custom failure message, so the CI log tells the engineer what to fix instead of making them decode a property path.

cfn-guard validate \
  --data template.yaml \
  --rules rules.guard \
  --show-summary fail
echo "exit code: $?"

template.yaml Status = FAIL
FAILED rules
rules.guard/s3_public_access_blocked    FAIL
---
Evaluation of rules rules.guard against data template.yaml
--
Property [/Resources/DataBucket/Properties/PublicAccessBlockConfiguration] is missing in data [template.yaml]. Error Message [Violation: every S3 bucket must block all four public-access settings.]
--
exit code: 19

Status = FAIL, the rule that tripped, and your own message. Exit code 19 is cfn-guard's signal that the data broke a rule. A different code, 5, means the run itself failed (a malformed rules file, a missing template), so your pipeline can tell a real policy violation apart from a broken job. --show-summary fail keeps the log quiet by printing only what failed. Add --type CFNTemplate and Guard prints the logical resource name (DataBucket) rather than the raw property path, which reads far better in a busy CI log.

Both tools judge the template's stated intent, not live infrastructure. Checking whether a running stack still matches its template is drift detection's job (cf-drift). Previewing what one specific change will do before you apply it is what change sets are for (cf-changesets). Scanning happens earlier than either, on text, before anything is built.

Deploy through a role that isn't you

The template passes review, and now you deploy it. The real question is whose hands are on the tools. Run the deploy under your own administrator login and CloudFormation can do anything you can do, which means so can anyone who steals that template or hijacks the pipeline that runs it. Hand the work to a locksmith carrying one key ring instead.

A service role is a dedicated identity (IAM, Identity and Access Management, is AWS's permission system) that CloudFormation assumes to do the actual creating, updating, and deleting. You pass it with --role-arn (ARN, Amazon Resource Name, is the unique ID string for any AWS resource) and scope it to exactly the permissions the template needs, nothing more. The people and pipelines that call CloudFormation then need only two things: permission to call CloudFormation, and iam:PassRole for that one role. The blast radius of a stolen pipeline token shrinks to whatever that single role can reach.

Two flags matter at this step. When the template creates permissions of its own (a role, a policy, an instance profile), CloudFormation refuses to move until you acknowledge it with --capabilities CAPABILITY_IAM, or CAPABILITY_NAMED_IAM when those resources carry names you chose. It is a deliberate speed bump, because creating IAM is how a template could quietly grant itself more power, so AWS makes you say out loud that you meant to. A template that uses a macro (a small program that rewrites the template into more resources before it is deployed) needs CAPABILITY_AUTO_EXPAND for the same reason.

Lock the load-bearing wall with a stack policy

A stack policy is the rule that says do not knock down the load-bearing wall during the remodel. It is a small JSON (JavaScript Object Notation, a plain text format for structured data) document attached to a running stack that controls what a future update may do to each resource. There is no default policy, and the absence of one means the default is permissive: an update may modify or replace anything, including the single database holding five years of customer records.

The danger is quiet. Some property changes force CloudFormation to replace a resource, meaning it builds a new one and deletes the old, and for a database that takes the data with it. A stack policy lets you forbid that one action on that one resource while leaving everything else free to update.

{
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "Update:*",
      "Principal": "*",
      "Resource": "*"
    },
    {
      "Effect": "Deny",
      "Action": ["Update:Replace", "Update:Delete"],
      "Principal": "*",
      "Resource": "LogicalResourceId/ProdDatabase"
    }
  ]
}

The first statement allows every update everywhere. The second carves out ProdDatabase and denies the two destructive actions: Update:Replace (build new, delete old) and Update:Delete (remove it outright). Principal is always the wildcard in a stack policy, because these documents do not name users, they gate the update operation itself.

aws cloudformation set-stack-policy \
  --stack-name prod-app \
  --stack-policy-body file://stack-policy.json
 
aws cloudformation deploy \
  --template-file template.yaml \
  --stack-name prod-app \
  --role-arn arn:aws:iam::111122223333:role/cfn-deploy-role \
  --capabilities CAPABILITY_NAMED_IAM

Waiting for changeset to be created..
Waiting for stack create/update to complete
Successfully created/updated stack - prod-app

From here on, any update that would replace or delete ProdDatabase is refused by CloudFormation before it lays a finger on the resource.

Resources:
  ProdDatabase:
    Type: AWS::RDS::DBInstance
    DeletionPolicy: Retain          # keep the DB if the whole stack is deleted
    UpdateReplacePolicy: Retain     # keep the old DB if a change forces a replace
    Properties:
      StorageEncrypted: true
      Engine: postgres
      DBInstanceClass: db.t3.medium

Prove the controls are on

A control you cannot see is a control you cannot trust. Scanning proves itself with the exit code your CI reads on every change, so gate on echo $? in the pipeline rather than skimming the log with your eyes. For the two deploy-time controls, ask the stack directly.

aws cloudformation get-stack-policy --stack-name prod-app

{
    "StackPolicyBody": "{\"Statement\":[{\"Effect\":\"Allow\",\"Action\":\"Update:*\",\"Principal\":\"*\",\"Resource\":\"*\"},{\"Effect\":\"Deny\",\"Action\":[\"Update:Replace\",\"Update:Delete\"],\"Principal\":\"*\",\"Resource\":\"LogicalResourceId/ProdDatabase\"}]}"
}

An empty response means no policy is attached and every resource is fair game for replacement. Check termination protection the same way, and turn it on if it is off.

aws cloudformation update-termination-protection \
  --stack-name prod-app \
  --enable-termination-protection

{
    "StackId": "arn:aws:cloudformation:us-east-1:111122223333:stack/prod-app/8a1b2c30-6b1a-11ee-9c2f-0e5a2f4b6c1d"
}

Make failure the default: scan with --fail-on-warnings, keep every deploy on its scoped role, and re-run get-stack-policy after each change to a protected stack. The day someone ships a template that would replace the database, you get a red pipeline or a refused update, not a 3 a.m. page about customer data that is no longer there.

Try this

Run gem install cfn-nag on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: a stack policy guards updates, and it blocks you too. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related