CloudFormation in CI/CD

Automate deploys; protect the pipeline.

The Plan Is The Gate

A deploy from a laptop is a promise nobody can check. You type a command, the terminal scrolls green, and production is different than it was ten seconds ago. Point at the wrong account or widen a security group by one line, and the first person to find out is whoever gets paged at 2 a.m. Continuous integration and continuous delivery (CI/CD, the practice of letting an automated pipeline test and ship changes instead of a person running commands by hand) turns that private promise into something a teammate can read, approve, and replay.

A good contractor does not start swinging a sledgehammer the moment you say 'renovate.' They walk the house and hand you an itemized quote: this wall comes down, that pipe moves, the water is off for a day. You read it, you sign it, then work starts. CloudFormation has the same object, called a change set. A change set is a preview. CloudFormation compares the new template you propose against the stack that is actually running and writes down every resource it would add, change, or replace. It applies nothing. Your infrastructure stays exactly as it was until you run a second, separate command to execute it.

That two-step shape is the whole design. On a pull request (a proposed change to the code, opened for review before it merges), the pipeline builds the change set and shows it. A person reads the quote. On merge, and only on merge, the pipeline executes it. The change set is the plan; the review before execute is the gate. Nobody deploys from a laptop against production, because the laptop was never holding the keys.

name: infra
on:
  pull_request:
  push:
    branches: [main]
 
permissions:
  id-token: write        # let the job mint an OIDC token
  contents: read
  pull-requests: write   # let it post the diff back to the PR
 
jobs:
  plan:
    if: github.event_name == 'pull_request'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789012:role/gha-prod-net-plan
          aws-region: us-east-1
      - run: pip install cfn-lint && cfn-lint template.yaml
      - run: cfn-guard validate -r security.guard -d template.yaml
      - run: |
          NAME="pr-${{ github.event.number }}"
          aws cloudformation create-change-set --stack-name prod-net \
            --change-set-name "$NAME" --template-body file://template.yaml \
            --role-arn arn:aws:iam::123456789012:role/cfn-prod-net-service \
            --capabilities CAPABILITY_NAMED_IAM
          aws cloudformation wait change-set-create-complete \
            --stack-name prod-net --change-set-name "$NAME"
          aws cloudformation describe-change-set --stack-name prod-net \
            --change-set-name "$NAME" --output table \
            --query 'Changes[].ResourceChange.{Action:Action,Id:LogicalResourceId,Type:ResourceType,Replace:Replacement}'
          # post the table to the PR as a comment (step omitted)
 
  deploy:
    if: github.event_name == 'push'
    runs-on: ubuntu-latest
    environment: production      # a human must approve before this job runs
    steps:
      - uses: actions/checkout@v4
      - uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789012:role/gha-prod-net-deploy
          aws-region: us-east-1
      - run: |
          NAME="deploy-${{ github.sha }}"
          aws cloudformation create-change-set --stack-name prod-net \
            --change-set-name "$NAME" --template-body file://template.yaml \
            --role-arn arn:aws:iam::123456789012:role/cfn-prod-net-service \
            --capabilities CAPABILITY_NAMED_IAM
          aws cloudformation wait change-set-create-complete \
            --stack-name prod-net --change-set-name "$NAME"
          aws cloudformation execute-change-set --stack-name prod-net \
            --change-set-name "$NAME"
          aws cloudformation wait stack-update-complete --stack-name prod-net

Read the plan job top to bottom. It checks out the code, then trades a short-lived identity token for temporary AWS (Amazon Web Services) credentials (more on that below). It lints and policy-scans the template, creates a change set named after the pull request number, waits for CloudFormation to finish building it, and prints the result as a table it can post back to the pull request. Notice what the plan job never does: it never calls execute-change-set. A pull request can propose anything and still touch nothing. On merge, the deploy job builds a fresh change set from the merged template and executes that one, so the plan it applies is the exact code a reviewer approved rather than a stale set from a branch that has since moved.

aws cloudformation create-change-set \
  --stack-name prod-net --change-set-name pr-42 \
  --template-body file://template.yaml \
  --role-arn arn:aws:iam::123456789012:role/cfn-prod-net-service \
  --capabilities CAPABILITY_NAMED_IAM
 
aws cloudformation wait change-set-create-complete \
  --stack-name prod-net --change-set-name pr-42

{
    "Id": "arn:aws:cloudformation:us-east-1:123456789012:changeSet/pr-42/6b8f2c1a-3d5e-4f70-9a2b-1c4d5e6f7a80",
    "StackId": "arn:aws:cloudformation:us-east-1:123456789012:stack/prod-net/9f1c2d3e-4a5b-6c7d-8e9f-0a1b2c3d4e5f"
}

The create command returns that identifier at once and the wait command prints nothing, blocking silently until the set is built. The change set now exists and has done nothing to your account. Ask CloudFormation what is inside it. The raw answer is a wall of JSON (JavaScript Object Notation, a text format for structured data), so pull out the four fields that decide whether the change is safe to apply.

aws cloudformation describe-change-set \
  --stack-name prod-net --change-set-name pr-42 --output table \
  --query 'Changes[].ResourceChange.{Action:Action,Id:LogicalResourceId,Type:ResourceType,Replace:Replacement}'

--------------------------------------------------------------------------------
|                               DescribeChangeSet                               |
+----------+---------------------+-----------+---------------------------------+
|  Action  |  Id                 |  Replace  |  Type                           |
+----------+---------------------+-----------+---------------------------------+
|  Add     |  FlowLogGroup       |  None     |  AWS::Logs::LogGroup            |
|  Modify  |  PublicSubnetA      |  False    |  AWS::EC2::Subnet               |
|  Modify  |  AppSecurityGroup   |  True     |  AWS::EC2::SecurityGroup        |
+----------+---------------------+-----------+---------------------------------+

Two Modify rows are unremarkable. The third is the one to stop on. Replace = True means CloudFormation cannot edit that resource in place; to apply your change it will delete the existing AppSecurityGroup and build a new one. For a security group that can mean a window where traffic is dropped. For a database or a volume it can mean the data is gone. Replacement is the most important column in a change set, and a reviewer who waves through an unexpected Replace = True on a stateful resource has approved an outage. This is also why the job waits for the set to build. If your template produces no changes at all, CloudFormation marks the change set FAILED, with a status reason that it found no changes to apply, and the waiter exits non-zero. Handle that case on purpose, or an empty diff reads as a broken pipeline.

Scanning Before AWS Ever Sees It

A building inspector checks the wiring against code before the drywall goes up, not after the house burns down. Two tools do that job for CloudFormation, and both run in the plan job as required checks (a pull request cannot merge until they pass). cfn-lint (CloudFormation lint, a static checker) reads the template and catches broken references, wrong property types, and impossible values without deploying anything. cfn-guard (a policy-as-code engine) checks the template against rules you write in near-plain language: every S3 (Simple Storage Service, AWS's file storage) bucket must be encrypted, no security group may open port 22 (the port for SSH, Secure Shell, remote login) to the internet.

# security.guard - policy as code, enforced on every pull request
rule s3_encryption_enabled {
    Resources.*[ Type == 'AWS::S3::Bucket' ] {
        Properties.BucketEncryption exists
    }
}
 
rule sg_no_public_ssh {
    Resources.*[ Type == 'AWS::EC2::SecurityGroup' ] {
        Properties.SecurityGroupIngress[ FromPort == 22 ] {
            CidrIp != '0.0.0.0/0'
        }
    }
}

cfn-lint template.yaml
cfn-guard validate --rules security.guard --data template.yaml

W2001 Parameter EnvName not used.
template.yaml:8:3
 
template.yaml Status = PASS
PASS rules
security.guard/s3_encryption_enabled    PASS
security.guard/sg_no_public_ssh         PASS

cfn-lint flagged an unused parameter and cfn-guard passed. In the pipeline these are wired as required status checks, so a red result greys out the merge button instead of emailing someone after the fact. One sharp edge lives here. GitHub Actions runs each step under a shell set to abort on the first error (bash with -e and pipefail). A warning makes cfn-lint exit non-zero, the same as an error does. Chain it behind another command with && and a harmless warning can fail the whole step. Decide on purpose whether warnings should block the merge, and set cfn-lint's exit behavior to match, rather than learning the rule from a red build on a Friday.

Who The Pipeline Is Allowed To Be

Here is the question that decides whether your pipeline is a convenience or a liability: how does GitHub prove to AWS that it may act on your account? The lazy answer is to paste an AWS access key (a long-lived username and password for a program) into a repository secret. That key is a house key mailed to the office and dropped in a drawer. Anyone who reaches the drawer, through a leaked log, a poisoned build dependency, a curious contractor, can copy it and walk into your account from anywhere, any time, until someone notices and changes the locks. OIDC (OpenID Connect, a standard way for one system to vouch for an identity to another) swaps the mailed key for a day pass. For each run, GitHub mints a signed token that says 'this is a job in the acme/prod-net repository,' AWS checks the signature, and its Security Token Service (STS) hands back keys that expire within the hour.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::123456789012:oidc-provider/token.actions.githubusercontent.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
        },
        "StringLike": {
          "token.actions.githubusercontent.com:sub": "repo:acme/prod-net:environment:production"
        }
      }
    }
  ]
}

The Condition block is the lock and the sub (subject, the line naming which job is asking) is its teeth. This deploy role hands credentials only to a job that entered the production environment, the same environment sitting behind the human approval gate. Read those together: the only way to become this role is to be a run that a person already approved. The plan role is pinned differently, to repo:acme/prod-net:pull_request, so it works on pull requests and nowhere else. Two identities, each cut to fit exactly one job.

One character turns that lock into a welcome mat. Writing the condition as repo:acme/prod-net:* feels harmless and is the most common OIDC mistake in the wild. That trailing wildcard matches every branch, every pull request, and every environment, so anyone who can push a throwaway branch can assume your production deploy role from their own workflow. Pin the sub to the exact branch or environment that may deploy. When you review someone else's trust policy, the StringLike value is the first line to read, and a trailing :* is a finding, not a detail.

What The Pipeline Is Allowed To Touch

Getting through the door is separate from being allowed to touch everything inside. The role GitHub assumes needs little: permission to make a few CloudFormation calls and permission to hand one specific role to CloudFormation. That second role, the service role, is what actually creates subnets and security groups. You already passed it, with --role-arn on create-change-set. The split matters because a compromised pipeline can then do only what the service role allows and not one action more. The IAM (Identity and Access Management, AWS's permission system) action iam:PassRole is the manager signing a slip that says this errand-runner may carry this one key, and CloudFormation, not the runner, is what turns it in the lock.

aws cloudformation describe-stacks --stack-name prod-net \
  --query 'Stacks[0].{Protected:EnableTerminationProtection,ServiceRole:RoleARN}'

{
    "Protected": true,
    "ServiceRole": "arn:aws:iam::123456789012:role/cfn-prod-net-service"
}

Two guardrails show up there. ServiceRole confirms CloudFormation acts as the scoped role rather than as whatever identity ran the deploy, so the stack cannot quietly exceed its permissions. EnableTerminationProtection set to true means a stray delete-stack call bounces off instead of taking production down. Neither is on by default. Both are one command to set, and worth setting the day the stack is born.

Proving The Guardrails Hold

None of this is real until you can check it from outside, the way an auditor would. Read the deploy role's trust policy and look at the one line that carries the weight.

aws iam get-role --role-name gha-prod-net-deploy \
  --query 'Role.AssumeRolePolicyDocument.Statement[0].Condition.StringLike'

{
    "token.actions.githubusercontent.com:sub": "repo:acme/prod-net:environment:production"
}

That value is pinned to an environment, not a wildcard, so the check passes. The last thread to pull is attribution. When the deploy role is assumed, its credentials carry a session name, and every CloudFormation call the pipeline makes lands in CloudTrail (AWS's audit log of who did what) under that assumed-role session. Set the role-session-name to carry the workflow run id, and a single ExecuteChangeSet event ties production's last change to an exact run, an exact commit, and the person who approved it. That is the gap between 'someone changed the firewall' and 'run 4821, commit a1b2c3d, approved by dana at 14:07.'

Wire the two-step gate first, then the two identities, then the checks that have to stay green. A change that comes out of this pipeline arrives with a plan a person read, a scan it had to pass, an approval a human gave, and a credential that expired an hour later. Test it the mean way: open a branch that is not main, point a workflow at your deploy role, and confirm AWS refuses to hand it credentials.

Try this

Run cfn-lint template.yaml on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: never 'fix' a fork by switching to pull_request_target. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related