Custom resources & macros

Extend CloudFormation beyond built-ins.

CloudFormation ships with a big catalog of resource types, one for every kind of thing AWS knows how to build: an EC2 (Elastic Compute Cloud, rentable virtual servers) instance, an S3 (Simple Storage Service, file storage) bucket, an IAM (Identity and Access Management, the permission system) role. The catalog is large. It is also finite. Sooner or later you need something that is not in it: a resource in a service so new that CloudFormation has not caught up, a setting over in a third-party tool like a monitoring provider or an outside DNS (Domain Name System, the internet's address book) host, or a bit of logic that has to run during the build itself, while CloudFormation is turning your template into a live stack (the set of resources it creates and manages as one unit): look up the newest machine image, seed a fresh database, generate a certificate. Two features close those gaps. They look similar, they fail in different ways, and one of them is far more dangerous than it looks, so it pays to know exactly which is which.

The Two Escape Hatches

Say your template is a set of building plans you hand to a builder. A custom resource is a phone call the builder makes partway through the job: "I cannot wire this part myself, let me ring the electrician and wait for them to report back." It runs your code while the stack is going up and folds whatever comes back into the stack as a managed piece. A macro is a different animal. It is an editor who rewrites the plans before the builder ever reads them. You write shorthand, the editor expands it into full instructions, and CloudFormation builds from the expanded copy. One does work during the build. The other changes the blueprint before the build starts. Mix them up and you reach for heavy machinery where a screwdriver would do.

Custom Resources: A Phone Call During the Build

CloudFormation calls a Lambda (a function that runs your code without a server you keep running) function that you point it at, once for each moment in the resource's life: Create, Update, and Delete. It hands the function a JSON (JavaScript Object Notation, plain-text structured data) message that says what it wants, plus one detail that matters more than any other: a ResponseURL, a single-use web address the function has to call back when it finishes. The function does its work, then sends SUCCESS or FAILED to that address along with any data it wants to return. That returned data becomes readable elsewhere in the template through GetAtt (get attribute, the intrinsic function that pulls a value off one resource so another can use it). If you have ever wanted the newest Amazon Linux image id without hard-coding it, this is one way to get it.

Resources:
  # The worker. Its code is inlined below as index.py; entry point index.handler.
  AmiLookupFn:
    Type: AWS::Lambda::Function
    Properties:
      Runtime: python3.12
      Handler: index.handler                   # module index, function handler
      Timeout: 30
      Role: !GetAtt AmiLookupRole.Arn
      Code:
        ZipFile: |          # inline code lands in a file named index.py
          ...
 
  # The custom resource. CloudFormation phones AmiLookupFn on
  # create, update, and delete.
  LatestAmi:
    Type: Custom::AmiLookup                   # Custom:: = your code, not a built-in
    Properties:
      ServiceToken: !GetAtt AmiLookupFn.Arn   # who to call
      Region: !Ref "AWS::Region"              # a prop; changing any prop re-runs the function
 
  AppServer:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: !GetAtt LatestAmi.Id           # the value the Lambda returned
      InstanceType: t3.micro

import cfnresponse   # ships automatically when you inline the code like this
import boto3
 
def handler(event, context):
    # Tearing the stack down? Nothing to undo. Answer and return,
    # or the delete hangs exactly like a bad create.
    if event["RequestType"] == "Delete":
        cfnresponse.send(event, context, cfnresponse.SUCCESS, {})
        return
 
    try:
        ec2 = boto3.client("ec2")
        images = ec2.describe_images(
            Owners=["amazon"],
            Filters=[{"Name": "name", "Values": ["al2023-ami-2023.*-x86_64"]}],
        )["Images"]
        newest = max(images, key=lambda i: i["CreationDate"])
        cfnresponse.send(event, context, cfnresponse.SUCCESS,
                         {"Id": newest["ImageId"]})
    except Exception as err:
        # Every failure path STILL has to send a response.
        cfnresponse.send(event, context, cfnresponse.FAILED, {},
                         reason=str(err))

The Type starts with Custom::, which is how CloudFormation knows this is your code and not a built-in. The ServiceToken is the ARN (Amazon Resource Name, the unique id of an AWS object) of the Lambda to call. You can point a custom resource at an SNS (Simple Notification Service, AWS's messaging system) topic instead of a Lambda, though Lambda is what nearly everyone uses. One quiet trap lives in that Region property: a custom resource only re-runs on a stack update if one of its properties changes. Change a property and the function runs again. Leave every property identical and CloudFormation skips the call and reuses the old answer, even if the real world has moved on.

Watching a Custom Resource Run

Because a custom resource is your code executing inside a stack operation, you get to watch it in two places. The stack's event stream shows the resource moving through its lifecycle, and the function's own log group in CloudWatch Logs (AWS's built-in logging service) shows what the code actually did.

aws cloudformation describe-stack-events --stack-name web \
  --query "StackEvents[?ResourceType=='Custom::AmiLookup'].[Timestamp,ResourceStatus]" \
  --output table

------------------------------------------------------
|                 DescribeStackEvents                |
+-----------------------------+----------------------+
|  2026-07-21T09:14:52.612Z   |  CREATE_COMPLETE     |
|  2026-07-21T09:14:29.184Z   |  CREATE_IN_PROGRESS  |
+-----------------------------+----------------------+

aws logs tail /aws/lambda/web-AmiLookupFn-1AB2C3D4E5F6 --since 5m --format short

2026-07-21T09:14:31 START RequestId: 7c2f0b9e-3d41-4a8e-9c2b-5e1f6a7d8b90 Version: $LATEST
2026-07-21T09:14:32 Response body:
2026-07-21T09:14:32 {"Status": "SUCCESS", "Reason": "See the details in CloudWatch Log Stream: 2026/07/21/[$LATEST]a1b2c3d4", "PhysicalResourceId": "2026/07/21/[$LATEST]a1b2c3d4", "StackId": "arn:aws:cloudformation:us-east-1:123456789012:stack/web/8f0e...", "RequestId": "b7d9c4e1-...", "LogicalResourceId": "LatestAmi", "NoEcho": false, "Data": {"Id": "ami-0e2c8caab1f9d3e77"}}
2026-07-21T09:14:32 Status code: 200
2026-07-21T09:14:32 END RequestId: 7c2f0b9e-3d41-4a8e-9c2b-5e1f6a7d8b90
2026-07-21T09:14:32 REPORT RequestId: 7c2f0b9e-3d41-4a8e-9c2b-5e1f6a7d8b90 Duration: 839.44 ms  Billed Duration: 840 ms  Memory Size: 128 MB  Max Memory Used: 84 MB

That Status code: 200 line is the one to look for. It is the cfnresponse helper confirming the answer reached the ResponseURL. No 200, no response, and the stack is still standing there holding the phone.

aws cloudformation describe-stack-events --stack-name web \
  --query "StackEvents[?ResourceStatus=='CREATE_FAILED'].ResourceStatusReason" \
  --output text

CloudFormation did not receive a response from your Custom Resource. Please check your logs for requestId [a3f18d2c-9b47-42e1-bd80-6c0f5e2a1c34]. If you are using the Python cfn-response module, you may need to update your Lambda function code so that CloudFormation can attach the correct response

Read that message closely. It is telling you the function ran but never called back, which points you at your own error handling, not at CloudFormation. The fix is almost always a missing cfnresponse.send on some branch you forgot about.

Macros Rewrite the Template Before It Deploys

A macro flips the order around. Instead of running during the build, it runs before it, taking your template as input and returning a rewritten template that CloudFormation then deploys. You register a Lambda as a macro and switch it on with a Transform line at the top of the file. The most widely used macro is AWS::Serverless, better known as SAM (Serverless Application Model). It lets you describe a function and the web endpoint that fronts it (a web address other programs call over HTTP, the web's request-and-response protocol) in a handful of lines, then expands that shorthand into the dozen underlying resources it really takes. Macros add a layer of indirection, so reach for one only when plain templating cannot express the pattern.

Transform: AWS::Serverless-2016-10-31   # switch the SAM macro on
 
Resources:
  Api:
    Type: AWS::Serverless::Function       # SAM shorthand
    Properties:
      Handler: index.handler
      Runtime: nodejs20.x
      Events:
        Http:
          Type: Api
          Properties:
            Path: /
            Method: get

Those ten-or-so lines do not deploy as written. The SAM macro rewrites them first. To see the rewrite, ask CloudFormation for the processed template, the version it actually built from.

aws cloudformation get-template --stack-name api \
  --template-stage Processed \
  --query TemplateBody \
  | jq '.Resources | to_entries | map({(.key): .value.Type}) | add'

{
  "Api": "AWS::Lambda::Function",
  "ApiRole": "AWS::IAM::Role",
  "ApiHttpPermissionProd": "AWS::Lambda::Permission",
  "ServerlessRestApi": "AWS::ApiGateway::RestApi",
  "ServerlessRestApiDeployment47e6f2a1c8": "AWS::ApiGateway::Deployment",
  "ServerlessRestApiProdStage": "AWS::ApiGateway::Stage"
}

One function in your file became six real resources, including an IAM role you never wrote by hand. Sit with that for a second. The macro decided what that role can do. If you did not read the expanded output, you deployed permissions you never saw. Writing your own macro is the same shape underneath: a Lambda that receives a template fragment and returns a changed one, registered as a resource so other templates can opt in with Transform.

Resources:
  # Register a Lambda as a reusable macro named "Uppercase".
  UppercaseMacro:
    Type: AWS::CloudFormation::Macro
    Properties:
      Name: Uppercase
      FunctionName: !GetAtt MacroFn.Arn
# Elsewhere, a template opts in with:  Transform: [Uppercase]

Always Deploy Through the Processed Template

Here is the security payoff, and it applies double to any macro you did not write yourself. The template in your repository is not what runs. The processed template is. A macro is a small supply chain: whoever controls that Lambda controls what your stack actually deploys, and a hostile or buggy one can quietly add an admin role, open a security group (a virtual firewall around your resources) to the whole internet, or wire in a resource that copies your data somewhere else. Your source file would look clean the entire time. So before you run a stack that uses a macro, generate a change set (a preview of every resource CloudFormation is about to add, change, or remove, shown before anything actually happens). Creating a change set forces CloudFormation to run the macros and lay out the real resource changes, so you approve what ships instead of what you wrote.

Two habits fall out of this. First, diff the Processed template against the Original stage whenever a macro is involved, because that diff is the only honest picture of what deploys. Second, lock down who can update the macro's Lambda, because updating that one function silently changes the outcome of every stack that transforms through it.

Keep the Blast Radius Small

Both features run code under an IAM role you attach, while the stack is being built, with whatever permissions you grant. Treat that role like a key to one room, not a key to the whole building. The AMI (Amazon Machine Image, the disk template a server boots from) lookup needs ec2:DescribeImages and permission to write its own logs, and nothing more. If someone tampers with the function, tight permissions are the difference between a wasted deploy and a breached account. Two more guardrails worth setting. Any data a custom resource returns is stored in the stack and readable by anyone who can describe it, so set NoEcho on sensitive values to keep them out of the console and the events. And remember that the least code you have to secure is the code you never wrote. Before reaching for a custom resource, check whether a built-in already covers it. The newest Amazon Linux image, for one, is published by AWS in Systems Manager Parameter Store (SSM Parameter Store, a shared store of named configuration values whose public entries any account can read), so a stack parameter of type AWS::SSM::Parameter::Value<AWS::EC2::Image::Id> pointed at that public path resolves the current id at deploy time with no code of yours at all.

When you have to pick between them, the question to ask is short. If you need CloudFormation to do something while it builds, reach for a custom resource. If you need to change what CloudFormation builds from, reach for a macro. Decide that before you write a line of code, and you will not hand arbitrary template-rewriting power to a problem that a single Lambda call would have solved.

Try this

Run aws logs tail /aws/lambda/web-AmiLookupFn-1AB2C3D4E5F6 --since 5m --format short on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: a silent custom resource freezes the whole stack. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related