StackSets: multi-account & region

Deploy one template everywhere.

A CloudFormation stack is like a single storefront: one template, one Amazon Web Services (AWS) account, one region. CloudFormation is AWS's infrastructure-as-code service, which means you describe the resources you want in a text file (the template), and it builds them and tracks them for you as one unit called a stack. A StackSet is the franchise headquarters. You write the master template once, then stamp identical outlets across dozens of accounts and every region you run in, all from one desk. The unit of reuse is the whole template, unchanged. You are not copying YAML (the plain-text format templates are written in) from account to account. You deploy one definition to many places and change it everywhere in lockstep. When a new account joins your organization (a group of AWS accounts managed together under the AWS Organizations service), headquarters can open that account's outlet on its own, before anyone even logs in.

This is how a platform or security team lands a baseline across a whole estate without opening a hundred consoles by hand: a central logging bucket, an AWS Config rule (AWS Config records how your resources are set up and flags the ones that drift away from policy), a break-glass admin role (the emergency login you keep for when normal access is gone), a cross-account audit role. Write it once, land it everywhere, prove it is everywhere from a single command. The flip side is blast radius. Whoever can change the StackSet changes every account at once, so the same lever that ships a good baseline ships a bad one every bit as fast.

Pick a Permission Model First

Before a StackSet can build anything in another account, it needs permission to reach in. The real question is who cuts the keys to every outlet. There are two models, and they are hard to swap once you are running.

The self-managed model has you cut and hand out the keys yourself. You create two IAM (Identity and Access Management, the AWS service that decides who is allowed to do what) roles: an administration role in the account that owns the StackSet, and an execution role that trusts that admin role inside every target account. It works even for accounts that belong to no organization, but you own all of that trust wiring, and every execution role is a standing cross-account door an attacker would be glad to find.

The service-managed model hands the key-cutting to AWS Organizations. You switch on trusted access once, target organizational units (OUs, the folders that group accounts inside an organization) instead of raw twelve-digit account numbers, and let auto-deployment open and close stacks as accounts join or leave those OUs. For targets inside an organization, service-managed is almost always the right call. Keep self-managed for accounts that live outside it.

Turning on trusted access is one call from the management account, and you can confirm it in one more.

# from the organization management account: enable trusted access, then confirm
aws cloudformation activate-organizations-access
aws cloudformation describe-organizations-access --query Status --output text

ENABLED

Author the Baseline, Then Register It

Here is the kind of template a security team actually ships. It creates a named cross-account audit role that your central security account can assume (temporarily take on) inside every member account, plus an SSM (AWS Systems Manager) parameter, a small named value in a shared store that other tools can read.

AWSTemplateFormatVersion: '2010-09-09'
Description: Org security baseline - cross-account audit role + log-level switch
 
Parameters:
  LogLevel:
    Type: String
    Default: INFO
    AllowedValues: [INFO, DEBUG]
 
Resources:
  SecurityAuditRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: org-security-audit          # a *named* resource -> needs CAPABILITY_NAMED_IAM
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              AWS: 'arn:aws:iam::111122223333:root'   # the central security account
            Action: 'sts:AssumeRole'
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/SecurityAudit'
 
  BaselineLogLevel:
    Type: AWS::SSM::Parameter
    Properties:
      Name: /baseline/log-level
      Type: String
      Value: !Ref LogLevel

Because the role has a fixed RoleName, CloudFormation will not build it unless you explicitly acknowledge that the template mints named IAM resources. That acknowledgement is the CAPABILITY_NAMED_IAM flag. It exists so a template cannot quietly create privileged, predictably named roles across your whole estate without someone saying yes on purpose.

aws cloudformation create-stack-set \
  --stack-set-name org-baseline \
  --template-body file://baseline.yaml \
  --permission-model SERVICE_MANAGED \
  --auto-deployment Enabled=true,RetainStacksOnAccountRemoval=true \
  --capabilities CAPABILITY_NAMED_IAM

{
    "StackSetId": "org-baseline:9f8e7d6c-5b4a-4c2d-1e0f-a1b2c3d4e5f6"
}

That call registers the definition and does nothing else. No account has the audit role yet. A StackSet on its own is a blueprint sitting in a drawer.

Instances Turn the Blueprint Into Stacks

Registering the set printed the franchise manual. Stack instances open the real outlets. Each instance is exactly one account-and-region pairing, and each one becomes a genuine, independent CloudFormation stack you can inspect, tag, and check on its own. You bind instances to targets, and because deployment targets accept a whole OU, one ou-abc1-2defghij covers every account beneath it today and every account that lands there next month.

aws cloudformation create-stack-instances \
  --stack-set-name org-baseline \
  --deployment-targets OrganizationalUnitIds=ou-abc1-2defghij \
  --regions us-east-1 eu-west-1 \
  --parameter-overrides ParameterKey=LogLevel,ParameterValue=DEBUG \
  --operation-preferences \
      RegionConcurrencyType=SEQUENTIAL,MaxConcurrentPercentage=25,FailureTolerancePercentage=10

{
    "OperationId": "8b3f1c2a-6d7e-4f90-a1b2-c3d4e5f60718"
}

Operation Preferences Are Your Blast Radius

That one call kicked off an operation, and the operation preferences decide whether it lands as a calm wave or a company-wide outage. RegionConcurrencyType defaults to SEQUENTIAL, so regions roll one after another: us-east-1 finishes before eu-west-1 begins. MaxConcurrentPercentage (or MaxConcurrentCount) sets how many accounts deploy at the same time within a region. FailureTolerancePercentage (or FailureToleranceCount) is the circuit breaker: once that many instances fail in a region, CloudFormation stops rolling the change forward instead of marching the same broken change into every account that is left. The defaults are cautious, which is exactly what you want when the blast radius is your entire company.

The --parameter-overrides flag lets one template flex to local values without ever forking it, so a noisy account can run LogLevel=DEBUG while the rest keep the INFO default. You also do not have to drive any of this from the organization's management account. Register a member account as a delegated administrator (a member account you authorize to manage StackSets on the organization's behalf) and run the same commands there with --call-as DELEGATED_ADMIN. That keeps the org root out of daily use and gives you a smaller, better-watched target.

Treat the right to call create-stack-set and update-stack-set as estate-wide, near-root power, because that is what it is. An attacker who gains it can push a template that drops an attacker-owned IAM role into every account in a single operation. Every StackSet action lands in CloudTrail (the AWS service that logs every API call, meaning every request made to AWS, along with who made it and when) under events like CreateStackInstances and UpdateStackSet. Alert on them, hand the permission to a short list of humans and pipelines, and run day to day through a delegated admin so the quiet management account makes any real activity stand out. Then build the habit of drift detection: aws cloudformation detect-stack-set-drift tells you when a live stack no longer matches its template, which is exactly what a hand-edit or a tamper looks like. On a security baseline, a DRIFTED instance is an incident lead, not a curiosity.

Roll Out Updates as a Canary

When the template changes, update-stack-set rolls the new version out to every existing instance under the same kind of rollout preferences. The safe way to ship a risky change is a canary, named for the bird miners carried underground to warn them of bad air: send the change to one account first, stop dead on the first failure, see what broke, then widen. One account pays for the mistake instead of all of them.

# canary: one account at a time, halt on the very first failure
aws cloudformation update-stack-set \
  --stack-set-name org-baseline \
  --template-body file://baseline.yaml \
  --capabilities CAPABILITY_NAMED_IAM \
  --operation-preferences MaxConcurrentCount=1,FailureToleranceCount=0

{
    "OperationId": "d90a7e11-2c33-4b5a-8e6f-77aa11bb22cc"
}

MaxConcurrentCount=1 is one account at a time. FailureToleranceCount=0 means the first failed instance stops everything. Note that you pass --capabilities CAPABILITY_NAMED_IAM again: an update that touches named IAM resources needs the same explicit yes the create did.

The OUTDATED Trap and the Account-Removal Trap

Two StackSet surprises bite people hard, and both grow from the same blind spot: a StackSet does not keep itself in sync for you. Say that canary hit a snag in one account and FailureToleranceCount=0 stopped the run. The accounts it already finished read CURRENT, meaning caught up with the template. Everything it never reached stays on the old template and reads OUTDATED, meaning left behind. It will not catch up on its own. And because a stack set runs only one operation at a time, you cannot fire a second update to sweep up the stragglers until the first one finishes. You check with one command.

aws cloudformation list-stack-instances \
  --stack-set-name org-baseline \
  --query 'Summaries[].[Account,Region,Status]' \
  --output table

---------------------------------------------
|            ListStackInstances             |
+----------------+-------------+------------+
|  444455556666  |  eu-west-1  |  CURRENT   |
|  444455556666  |  us-east-1  |  CURRENT   |
|  555566667777  |  eu-west-1  |  OUTDATED  |
|  555566667777  |  us-east-1  |  OUTDATED  |
+----------------+-------------+------------+

Two accounts, four instances, two of them still running yesterday's template. Read this after every rollout. CURRENT across the board is the only real done; anything OUTDATED or FAILED means run the operation again to bring the rest into line.

Try this

Run aws cloudformation activate-organizations-access on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: the Permission Model Is Set in Concrete. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related