Your pipeline runs the put-targets call from this lesson against gd-critical and the step goes green. A severity 8 GuardDuty finding then lands on the bus and the ir-runbook state machine never starts. What do you look at first?

Look at FailedEntryCount in the response, because a rejected target does not fail the command. Put-targets reports each entry separately and the CLI still exits 0 when one of them is refused, usually for a bad ARN or a missing permission. The example output shows FailedEntryCount 0 and FailedEntries empty, which is what your pipeline should assert instead of trusting the exit code.

The gd-critical rule already carries five targets. A platform team asks to run their own Lambda on the same GuardDuty findings. What do you tell them?

Let them add a second rule matching the same pattern, since the bus allows 300 rules. Consumers scale at the bus, not inside one rule. Their rule gets its own five targets, its own retry policy, and its own DLQ, and nobody edits or redeploys gd-critical.

A teammate extends ir-runbook.asl.json with a Choice state that branches on severity, and pastes the Quarantine state's Retry block onto it so every state gets a second chance. What happens?

The deploy is rejected, since only states that call something may carry Retry. Retry and Catch belong to Task, Map, and Parallel, the three state types that invoke something capable of failing. A Choice, Pass, or Wait state with a Retry makes the whole state machine invalid at deploy time.

Courses AWS DevOps Engineer ProfessionalEventBridge & orchestration

EventBridge & orchestration

Route events to automated responses; Step Functions.

Advanced30 min · lesson 12 of 15

A big office building has a mailroom. Envelopes arrive from hundreds of departments, a sorter reads the address on each one, and copies drop into the right pigeonholes. The sender never finds out who eventually reads it. Amazon EventBridge is that mailroom for AWS (Amazon Web Services). An event is a small JSON (JavaScript Object Notation, a plain-text data format) record saying *something happened*: 'a GuardDuty finding fired', 'a pipeline stage failed'. A bus is the inbound tray that catches events. A rule is the sorter's matching instruction. A target is the pigeonhole, which is the Lambda function, queue, or workflow that reacts. Producers emit, the bus routes, consumers react, and none of them are wired directly to each other. That decoupling is the whole point. Adding a new reaction is a rule change, not a code change to the producer.

What an event looks like, and how a rule matches it

Every event arrives in the same envelope: source, detail-type, account, region, time, resources, and a detail object whose shape depends on whichever service sent it. Each account already has a default event bus, and it receives every AWS service event for free. You create custom buses for your own applications, and AWS hands you partner buses for SaaS (software as a service) sources like Datadog or PagerDuty. A rule matches using an event pattern, and that pattern is a *structural subset* match. Only the keys you name have to be present. An array of values means 'any of these'. Anything beyond an exact string comparison uses a content filter: prefix, suffix, anything-but, numeric comparisons, exists, wildcard, $or, and cidr, which tests an address against a CIDR (Classless Inter-Domain Routing) block. The rule below fires only on GuardDuty findings, Amazon's threat detection service, with a severity of 7 or higher.

put-rule.sh

# A trimmed GuardDuty event that lands on the default bus:
#   { "source": "aws.guardduty", "detail-type": "GuardDuty Finding",
#     "detail": { "severity": 8, "type": "UnauthorizedAccess:EC2/SSHBruteForce", ... } }
 
aws events put-rule --name gd-critical --state ENABLED \
  --event-pattern '{
    "source": ["aws.guardduty"],
    "detail-type": ["GuardDuty Finding"],
    "detail": { "severity": [ { "numeric": [">=", 7] } ] }
  }'
 
# {
#     "RuleArn": "arn:aws:events:us-east-1:111122223333:rule/gd-critical"
# }

Targets, retries, and why one event can fire a target twice

One rule reaches five targets at most. That ceiling is hard, and no support ticket will raise it. To go wider, point one of those five at an SNS (Simple Notification Service) topic or a Step Functions state machine and let it spread from there. Some targets need a RoleArn, the ARN (Amazon Resource Name, the unique identifier of an AWS resource) of an IAM (Identity and Access Management) role that EventBridge assumes on your behalf. Step Functions, SSM (Systems Manager) Automation, and cross-account buses all work that way. Lambda, SNS, and SQS (Simple Queue Service) go the other direction and rely on a resource policy attached to the target itself. The delivery model is the part that bites people. EventBridge is at-least-once with best-effort ordering, so a target can be invoked more than once and events can turn up out of order. That means your targets have to be idempotent, which is a long word for 'running it twice leaves the same end state'. Always attach a RetryPolicy and a dead-letter queue (DLQ), a holding pen for events that could not be delivered, and use an input transformer to reshape the event before the target ever sees it.

put-targets.sh

aws events put-targets --rule gd-critical --targets '[
  {
    "Id": "isolate",
    "Arn": "arn:aws:states:us-east-1:111122223333:stateMachine:ir-runbook",
    "RoleArn": "arn:aws:iam::111122223333:role/eb-invoke-sfn",
    "RetryPolicy": { "MaximumEventAgeInSeconds": 3600, "MaximumRetryAttempts": 20 },
    "DeadLetterConfig": { "Arn": "arn:aws:sqs:us-east-1:111122223333:eb-gd-dlq" }
  },
  { "Id": "page", "Arn": "arn:aws:sns:us-east-1:111122223333:oncall" }
]'
 
# {
#     "FailedEntryCount": 0,
#     "FailedEntries": []
# }
# FailedEntryCount > 0 means a target was rejected (bad ARN or missing permission) —
# always assert it is 0 in CI, the CLI still exits 0 on partial failure.

Without a dead-letter queue, failed events vanish quietly

When a target keeps failing, EventBridge retries it for up to 24 hours or 185 attempts, then throws the event away. CloudWatch does count the damage, in FailedInvocations and InvocationsSentToDlq, but a counter nobody alarms on pages nobody, and it never tells you which event went missing. On a rule whose job is isolating a compromised resource, that discarded event is a remediation that never ran. Attach a DeadLetterConfig SQS queue to every target that matters, alarm on its ApproximateNumberOfMessagesVisible metric, and alarm on FailedInvocations while you are there. The queue is the only place the failed event itself survives, which is the difference between knowing a delivery broke and being able to run it again.

Step Functions, for when one reaction is not enough

Orchestration is a grand word for a recipe. Do this step, then that one, in a fixed order, with a plan for what happens when a step falls over. A single Lambda handles a single action. A real incident response reads more like a checklist: quarantine the instance, snapshot its volume, tag it, page the on-call, in that sequence. For that, the EventBridge rule triggers a Step Functions state machine rather than a lone function. State machines are written in Amazon States Language, a JSON dialect where Task states call services, Choice branches on a condition, Map fans out over a list, and Parallel runs branches side by side. Retry and Catch blocks attach only to Task, Map, and Parallel states, the three kinds that call something which can fail. Put a Retry on a Choice, Pass, or Wait state and the state machine is rejected as invalid at deploy time. Two flavours exist. Standard workflows are exactly-once at the level of the whole execution, so one trigger never starts the same run twice, though a Task carrying a Retry clearly does run more than once and whatever it calls has to tolerate that. They last up to a year and cost \$0.025 per 1,000 state transitions. Express workflows are at-least-once, cap out at 5 minutes, and bill per request plus duration, which suits high-volume event processing.

ir-runbook.sh

# ir-runbook.asl.json — retry throttles, then route any failure to a human
{
  "StartAt": "Quarantine",
  "States": {
    "Quarantine": {
      "Type": "Task",
      "Resource": "arn:aws:states:::aws-sdk:ec2:modifyInstanceAttribute",
      "Parameters": {
        "InstanceId.$": "$.detail.resource.instanceDetails.instanceId",
        "Groups": ["sg-0quarantine"]
      },
      "Retry": [ { "ErrorEquals": ["States.ALL"], "IntervalSeconds": 2, "MaxAttempts": 3, "BackoffRate": 2.0 } ],
      "Catch": [ { "ErrorEquals": ["States.ALL"], "Next": "AlertHuman" } ],
      "End": true
    },
    "AlertHuman": {
      "Type": "Task", "Resource": "arn:aws:states:::sns:publish",
      "Parameters": { "TopicArn": "arn:aws:sns:us-east-1:111122223333:oncall", "Message.$": "$.Cause" },
      "End": true
    }
  }
}
 
aws stepfunctions describe-execution \
  --execution-arn arn:aws:states:us-east-1:111122223333:execution:ir-runbook:gd-7f3c \
  --query '{status:status, start:startDate, stop:stopDate}'
 
# {
#     "status": "SUCCEEDED",
#     "start": "2026-07-14T09:12:03.418000-04:00",
#     "stop":  "2026-07-14T09:12:19.902000-04:00"
# }

Two clocks: Scheduler and the older cron rules

AWS gives you two ways to run something on a clock. The older one is a scheduled rule: an ordinary rule sitting on the event bus whose trigger is a cron() or rate() expression instead of an event pattern. The newer one, EventBridge Scheduler, is a separate service built for this job alone. It runs one-time schedules with at() and recurring ones with rate() and cron(). It offers flexible time windows, which jitter each start time inside a window so ten thousand schedules do not all fire on the same second. It carries its own retry settings and its own DLQ, it can target more than 270 AWS services, and it scales to millions of schedules. Reach for Scheduler on anything new. The example below prunes old EBS (Elastic Block Store) snapshots every night at 03:00 UTC (Coordinated Universal Time) with a 15-minute flexible window.

create-schedule.sh

aws scheduler create-schedule --name nightly-ebs-snapshot-prune \
  --schedule-expression 'cron(0 3 * * ? *)' \
  --schedule-expression-timezone 'UTC' \
  --flexible-time-window '{ "Mode": "FLEXIBLE", "MaximumWindowInMinutes": 15 }' \
  --target '{
    "Arn": "arn:aws:states:us-east-1:111122223333:stateMachine:prune-snapshots",
    "RoleArn": "arn:aws:iam::111122223333:role/scheduler-invoke",
    "RetryPolicy": { "MaximumEventAgeInSeconds": 3600, "MaximumRetryAttempts": 5 }
  }'
 
# {
#     "ScheduleArn": "arn:aws:scheduler:us-east-1:111122223333:schedule/default/nightly-ebs-snapshot-prune"
# }

Limits, the bill, and locking the bus down

Learn the ceilings before production finds them for you. Quotas: 5 targets per rule (hard), 300 rules per bus (soft, so support can raise it), 100 event buses per account, a 256 KB maximum event size, up to 10 entries in one PutEvents batch, and a PutEvents rate ceiling that runs from 10,000 per second in the largest Regions (us-east-1, us-west-2, eu-west-1) down to a few hundred per second in the smallest, all adjustable. Cost: matching AWS service events on the default bus is free. Custom, partner, and cross-account events cost \$1.00 per million, metered in 64 KB chunks, so one 200 KB event bills as four. Scheduler charges \$1.00 per million invocations and gives you 14 million free every month. Hardening: put a resource-based policy on every custom bus so only accounts you trust can call PutEvents, give each target its own least-privilege IAM role instead of one shared role, and use the input transformer to forward only the fields a target actually reads rather than the whole event.

EventBridge routing fabric

Producers (event sources)

AWS services

GuardDuty, Config, CloudWatch alarms, CodePipeline (free on the default bus)

Your applications

PutEvents to a custom bus

SaaS partners

Datadog, PagerDuty via a partner bus

Router (the event bus)

Rules + event patterns

structural subset match, content filters

Retry + DLQ

at-least-once, up to 24h / 185 attempts

Input transformer

reshape the event before delivery

Targets (consumers)

Direct actions

Lambda, SNS, SQS, SSM Automation (max 5 per rule)

Orchestration

Step Functions state machine: retry, catch, branch

Scheduler

cron / rate / at() driven invocations

One event, many independent reactions. You add a consumer by adding a rule, never by touching the producer.

The decoupling pays for itself the first time another team wants the same events. They write a rule on the bus and start receiving them. Nobody edits the producer, nobody redeploys it, and nobody books a change window. A new consumer costs one rule and no coordination, which is how one GuardDuty finding ends up in a ticket queue, a log archive, and your quarantine workflow without those three teams ever meeting.

Keep the two clocks straight in your head. Scheduler is the modern cron for anything driven by time. Rules still win when the trigger is the shape of an event rather than an hour of the day. And when a response needs a person in the loop, a Step Functions workflow can hold at a state that waits for approval, retrying the automated steps around it and only continuing once a human says go.

Cross-account routing has one gotcha that catches everyone. A bus in the security account will not accept anything from your workload accounts until you attach a resource policy naming them. Skip that and your 'central security bus' is a private diary: you can write in it, nobody else can.

Design for duplicates, because you will get them. A remediation that runs a second time should be boring. A charge that runs a second time is a refund, and a page that fires a second time is an on-call engineer who starts ignoring pages. Exactly-once is a wish. A cheap check at the top of your handler is a plan.

Input transformers keep targets dumb in the good way. Cut an event down to the three fields a Lambda actually reads, and that function stops carrying a parser for CloudTrail-shaped JSON monsters. Smaller payloads also make DLQ triage readable at 3am, when you are squinting at one stuck message trying to work out what broke.

Archive and replay are the undo button. An archive keeps a copy of every event a bus received, for a retention period you pick. A replay pushes a chosen time range of those events back through your rules. A consumer that was down during Tuesday's burst and a Lambda with a bug you fixed on Wednesday both end the same way: you replay Tuesday afternoon instead of writing an apology. Turn archive on for buses that carry money or security findings while nothing is wrong, because you cannot archive the past.

Events are a contract, the same as an API (application programming interface). A schema registry records the shape each producer sends, and a versioned detail-type lets a consumer declare which shape it understands. Without that, a producer quietly renames a field, one consumer keeps working, the other three go silent, and nobody notices until the weekly report comes back empty. Break the contract on purpose with a new detail-type instead.

Try this

List the rules on the default bus, then look at what gd-critical actually points to. Run the put-rule and put-targets commands from earlier first, or the second command answers with ResourceNotFoundException. If you have a lab account handy, push a custom event onto the bus and confirm it was accepted.

terminal

aws events list-rules --event-bus-name default --query 'Rules[:5].{Name:Name,State:State}' --output table
aws events list-targets-by-rule --rule gd-critical --query 'Targets[].{Id:Id,Arn:Arn}' --output table
aws events put-events --entries '[{"Source":"lab.app","DetailType":"OrderPlaced","Detail":"{\"id\":\"o-1\"}","EventBusName":"default"}]' \
  --query '{Failed:FailedEntryCount,Entries:Entries}' --output json

output

gd-critical | ENABLED
Id=isolate | arn:aws:states:us-east-1:111122223333:stateMachine:ir-runbook
Id=page    | arn:aws:sns:us-east-1:111122223333:oncall
{"Failed":0,"Entries":[{"EventId":"a1b2c3d4-..."}]}

Takeaway

Events let reactions attach themselves to what happened, with nobody hand-wiring a producer to a consumer. Two habits keep that honest: make every consumer safe to run twice, and give every failed delivery a dead-letter queue to land in.

Next: wire one CloudWatch alarm or GuardDuty finding to a Lambda or Step Functions target, then fire a synthetic event and prove it arrives.

The next lesson, Progressive delivery, points this same machinery at deployments. A CloudWatch alarm emits an event, a rule catches it, and a Step Functions workflow shifts traffic back or lets the release roll further without waking anyone. Same buses, same rules, same idempotency habits. The only difference is that the thing being remediated is your own last deploy.

X-Ray & distributed tracingProgressive delivery

Global infrastructureAWS Solutions Architect Associate Microsoft Entra IDAzure Administrator Associate Azure Pipelines & CIAzure DevOps Engineer Professional