A payments API has to be serving again within a few minutes and can afford to lose essentially no data. You want the cheapest option that still meets both targets. Which DR strategy fits?

Warm standby. Warm standby keeps a smaller copy of the whole stack running all the time, fed by continuous replication, so you scale it up and repoint DNS in minutes with almost no data lost. Backup-and-restore and pilot light both take too long, and active-active buys speed this workload does not need, at roughly double the bill.

You are building a self-healing Auto Scaling group (ASG) behind a load balancer. Why does setting --health-check-type to ELB instead of EC2 matter so much?

Healthy then means the application is answering through the target group, so an instance that boots fine but fails the application's health check gets terminated and replaced. With ELB health checks, healthy means the application is answering through the target group, so a box that boots but serves errors is replaced. EC2 checks only confirm the machine started, which a completely broken app passes without trouble.

During a disaster-recovery test, Route 53 correctly starts handing out the standby Region's record, yet plenty of clients keep hitting the failed primary for several minutes and you blow your RTO (recovery time objective). What is the best fix before the next test?

Lower the record's TTL ahead of time, to 60 seconds for example, so resolvers stop holding the old answer as long. Route 53 changes the answer immediately, but clients and resolvers cache it for the length of the TTL, so 300 or 3600 seconds keeps traffic pointed at the dead primary. Set it to about 60 seconds in advance, because you cannot shrink it fast enough once the incident is under way.

Courses AWS DevOps Engineer ProfessionalHA & DR automation

HA & DR automation

Self-healing, backup, tested failover, RTO/RPO.

Advanced30 min · lesson 14 of 15

A fire drill nobody has ever run is not a safety plan. It is a hope. High availability and disaster recovery work exactly the same way. Drawing a resilient architecture is the easy half; a recovery path you have never exercised is a diagram on a wall. At this level the job is to write recovery down as code so it runs identically every time, measured and rehearsed, instead of being reassembled by hand at 3 a.m. while money drains out the door. HA (high availability, meaning the system shrugs off everyday failures on its own) absorbs a dead instance or a lost Availability Zone with nobody watching. DR (disaster recovery, meaning you can come back somewhere else after a big event) covers the rare, ugly ones, like a whole AWS Region going sideways. The rest of this lesson turns both into commands.

The two numbers that decide everything else

Two numbers drive every resilience decision, and you pick them before you pick a single service. RPO (recovery point objective) is how much data you can afford to lose, counted in time. A word processor that autosaves every five minutes has a five-minute RPO: pull the plug and at most five minutes of typing is gone. RTO (recovery time objective) is how long you can stay down before you are serving traffic again. RPO is a copying problem, about how fresh the copy sitting in the safe place is. RTO is a speed problem, about how fast you can bring that copy online and point users at it. Both get expensive very quickly as they approach zero, so you set them per workload from real business impact, never as one company-wide rule. A nightly reporting job lives happily with a 24-hour RPO and an 8-hour RTO. A payments ledger measures both in seconds. Every command below exists to hit one of those two numbers.

Four ways to do DR, and what each one actually costs

AWS lays disaster recovery out on a curve: the closer to zero you push RTO and RPO, the more you pay every single month, disaster or no disaster. Backup-and-restore is the cheap end. Keep backups, copy them to a second Region, rebuild when you need to, with recovery time and data loss both measured in hours. Pilot light keeps the core alive and the rest switched off. The database replicates continuously, the application tier is provisioned but dark, and you start and scale it during failover, which puts recovery in the tens of minutes for very little steady-state cost. Warm standby runs a complete copy of the stack in the DR Region all the time, only smaller, fed by that same continuous replication. On failover you scale it up, repoint DNS (the internet's address book, which turns a name into an address), and you are back in minutes with almost no data lost. Multi-site active-active serves live traffic from both Regions at once, so recovery time and data loss both sit near zero, but it costs the most and is the hardest thing on this list to build, because writes landing in two Regions have to be reconciled with each other. The usual mistake is buying too much: running active-active for a workload whose worst possible hour of downtime costs less than the second Region's bill.

Choosing a DR strategy from your RTO/RPO target

Every step toward zero costs more. Buy the cheapest tier that still meets the business target.

Start with the free part: a stack that heals itself

Before you spend a dollar on disaster recovery, take the high availability that is nearly free. The shape is an Auto Scaling group (ASG, a controller that keeps a fleet at the size you declared) spread across three Availability Zones behind a load balancer, with the group's idea of health handed over to the target group. An instance that fails the application's health check gets terminated and replaced with no ticket, no page, no human. Multi-AZ RDS (Relational Database Service running in two Availability Zones) or Aurora adds a hot standby that takes over by itself. The whole thing behaves like a thermostat: you state the temperature you want and the controller keeps nudging the room back toward it. The flag that makes this real is --health-check-type set to ELB (the load balancer) rather than EC2 (the virtual machine). ELB means healthy is defined by your application answering through the load balancer. EC2 means healthy is defined by the machine having booted, which is a much lower bar and quietly hides broken apps.

codify-ha-baseline.sh

# A self-healing ASG across 3 AZs, health-checked by the ALB target group
aws autoscaling create-auto-scaling-group \
  --auto-scaling-group-name web-asg \
  --launch-template LaunchTemplateId=lt-0abc123def456,Version='$Latest' \
  --min-size 3 --max-size 12 --desired-capacity 3 \
  --health-check-type ELB --health-check-grace-period 120 \
  --vpc-zone-identifier "subnet-0a1b,subnet-0c2d,subnet-0e3f" \
  --target-group-arns arn:aws:elasticloadbalancing:us-east-1:111122223333:targetgroup/web-tg/abc123
# (success is silent; exit code 0)
 
# Confirm capacity is spread and healthy — the controller keeps it that way
aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names web-asg \
  --query 'AutoScalingGroups[0].Instances[].{AZ:AvailabilityZone,Health:HealthStatus,State:LifecycleState}' \
  --output table
# -------------------------------------------
# |        DescribeAutoScalingGroups        |
# +-------------+----------+----------------+
# |     AZ      |  Health  |     State      |
# +-------------+----------+----------------+
# |  us-east-1a |  Healthy |  InService     |
# |  us-east-1b |  Healthy |  InService     |
# |  us-east-1c |  Healthy |  InService     |
# +-------------+----------+----------------+

Hitting your RPO: one backup policy, copied out of the Region

Your RPO is only as good as the freshest copy sitting outside the blast radius. AWS Backup gives you one policy-driven place to schedule backups, set how long they live, and copy them into another Region or another account, covering EBS volumes (Elastic Block Store, the disks attached to instances), RDS databases, DynamoDB tables, EFS file systems (Elastic File System) and more, without a pile of per-service scripts. When an hour of loss is too much, stop snapshotting and start streaming. Aurora Global Database ships writes to a secondary Region with lag usually under a second, S3 Cross-Region Replication copies objects across as they land, and DynamoDB global tables keep the same rows writable in several Regions at once. Keep an eye on the bill and the limits. Cross-Region copies pay inter-Region data transfer plus storage in *both* Regions, every warm replica is billed whether or not disaster ever arrives, and AWS Backup enforces per-account, per-Region quotas that a fleet-wide backup window will cheerfully run straight into.

rpo-aws-backup-plan.sh

# Hourly backups copied to us-west-2 — a cross-Region RPO of ~1 hour
aws backup create-backup-plan --backup-plan '{
  "BackupPlanName": "prod-critical",
  "Rules": [{
    "RuleName": "hourly-cross-region",
    "TargetBackupVaultName": "Default",
    "ScheduleExpression": "cron(0 * ? * * *)",
    "StartWindowMinutes": 60,
    "Lifecycle": {"DeleteAfterDays": 35},
    "CopyActions": [{
      "DestinationBackupVaultArn": "arn:aws:backup:us-west-2:111122223333:backup-vault:Default",
      "Lifecycle": {"DeleteAfterDays": 35}
    }]
  }]
}'
# {
#     "BackupPlanId": "a1b2c3d4-5e6f-7a8b-9c0d-1e2f3a4b5c6d",
#     "BackupPlanArn": "arn:aws:backup:us-east-1:111122223333:backup-plan:a1b2c3d4-...",
#     "CreationDate": "2026-07-14T09:12:04.512000-04:00",
#     "VersionId": "Zjc4Mtk...=="
# }

Automate the failover, or you will never hit your RTO

Recovery time is where automation pays for itself. A perfectly fresh copy of your data in the DR Region is worth nothing if reaching it means one tired person remembering twelve steps in the right order. Two pieces carry the weight. Route 53 failover routing keeps health-checking the primary endpoint and, the moment it fails, starts handing out the DR endpoint instead, which makes DNS your traffic switch. The heavier work belongs in an SSM Automation runbook (Systems Manager, the AWS service that runs stored operational documents for you): promote the Aurora Global secondary to a standalone writer, scale the warm standby up, flip the feature flags. Because it is a document rather than a wiki page, you can version it, run it against staging, replay the log of every step afterwards, and test it on a quiet Tuesday. Recovery becomes one auditable API call.

automate-failover.sh

# Route 53 already evaluates the primary's health and serves the DR record on failure
aws route53 get-health-check-status --health-check-id 3e8f1a2b-primary
# {
#     "HealthCheckObservations": [
#         {"Region": "us-east-1", "StatusReport": {"Status": "Failure: Connection timed out"}},
#         {"Region": "us-west-2", "StatusReport": {"Status": "Success: HTTP Status Code 200, OK"}}
#     ]
# }
 
# Promote the DR Region with a tested, idempotent SSM Automation runbook (not a wiki page)
aws ssm start-automation-execution \
  --document-name PromoteDrRegion \
  --parameters GlobalClusterId=prod-global SecondaryClusterArn=arn:aws:rds:us-west-2:111122223333:cluster:prod-dr
# {
#     "AutomationExecutionId": "b7c1e2f0-4a2b-4c3d-9e8f-0123456789ab"
# }
 
# Watch it converge to Success
aws ssm get-automation-execution --automation-execution-id b7c1e2f0-4a2b-4c3d-9e8f-0123456789ab \
  --query 'AutomationExecution.{Status:AutomationExecutionStatus,Step:CurrentStepName}'
# {
#     "Status": "Success",
#     "Step": "FailoverGlobalCluster"
# }

Fix your DNS TTL and your DR capacity now, not during the incident

Two things quietly ruin real failovers. The first is caching. Route 53 changes the record instantly, but every client and resolver on the internet keeps the old answer until the TTL (time to live, the number of seconds a DNS answer may be cached) runs out. A 300-second TTL, or worse a 3600-second one, means minutes of traffic still hammering a primary that is already dead. Drop it to 60 seconds well in advance, because you cannot shrink it fast enough once the incident has started. The second is capacity. A pilot-light or thinly scaled DR stack has to grow into full production load in a hurry, and scaling up from near zero can eat your entire RTO or hit a wall: an InsufficientInstanceCapacity error, or a per-Region service quota ceiling, in a Region that has just absorbed everybody else's failover as well. Pre-provision, use capacity reservations, or pay for a warm standby on the workloads whose RTO you actually mean.

Prove it with fault injection, because an untested recovery is a guess

A recovery path you have never run will break the day you need it. The runbook is stale. An IAM role (Identity and Access Management, the set of permissions the automation runs as) was never created in the DR Region. Replication fell behind six weeks ago and nobody looked. AWS Fault Injection Service (FIS, a service that breaks your own systems on purpose, under control) turns that discovery into a scheduled experiment. You write a template that says stop every instance in one Availability Zone, or add network latency, or fail the primary database. You attach *stop conditions*, which are CloudWatch alarms that abort the run the second the damage grows past what you planned. Then you start it, measure the RTO and RPO you actually got against the ones you promised the business, and close the gap in daylight with the full team watching, rather than during a real Region event. Rehearsing on a schedule is the one habit that separates a design that looks reliable from one that is.

rehearse-with-fis.sh

# Rehearse an AZ failure and confirm the ASG self-heals inside the RTO budget
aws fis start-experiment --experiment-template-id EXT5AbCdEf1234567
# {
#     "experiment": {
#         "id": "EXPabc123def456",
#         "experimentTemplateId": "EXT5AbCdEf1234567",
#         "state": {"status": "initiating", "reason": "Experiment is initiating."},
#         "startTime": "2026-07-14T13:40:11-04:00"
#     }
# }
 
# Poll until it finishes, then verify capacity recovered before the stop conditions fired
aws fis get-experiment --id EXPabc123def456 --query 'experiment.state'
# {
#     "status": "completed",
#     "reason": "Experiment completed."
# }

A backup plan with no restore test is a folder of files you hope are readable. Put AWS Backup inventory jobs and real restore drills on the calendar right next to your game days. A pilot light nobody has lit in a year will not light on the day it matters.

Route 53 health checks and failover records are the standard cutover tools, and the standard mistake is a health check that only notices when the network card dies. Point it at a URL that exercises the application, so it goes red when the app is broken and the server is still cheerfully accepting connections.

Aim your fault injection experiments at the exact failures your RTO assumes you will survive: losing an Availability Zone, a dependency that turns slow, a database that stops answering. Run them in a non-production account first.

Try this

Find out what your account is actually promising. List the backup plans, pull a few recent recovery points out of the default vault, and, if you use DNS failover, list your health checks.

terminal

aws backup list-backup-plans --query 'BackupPlansList[].{Name:BackupPlanName,Id:BackupPlanId}' --output table
aws backup list-recovery-points-by-backup-vault --backup-vault-name Default \
  --query 'RecoveryPoints[:3].{Arn:RecoveryPointArn,Status:Status,Created:CreationDate}' --output table
aws route53 list-health-checks --query 'HealthChecks[].{Id:Id,Type:HealthCheckConfig.Type}' --output table

output

daily-prod | aa11-bb22
arn:aws:backup:...:recovery-point:abcd | COMPLETED | 2026-07-23T03:00:00Z
hc-123 | HTTPS

Takeaway

High availability is the part that happens without you, inside one Region. Disaster recovery is the part you rehearse across Regions, and the only honest measure of it is the RTO and RPO you clocked on the last drill. A backup you have never restored is an opinion.

This month, restore one production backup into a scratch account and put a stopwatch on it. Then compare that number with the RTO you wrote down.

Automating and rehearsing failover shrinks the technical half of recovery to a procedure you can time with a stopwatch. The human half is still open: who declares the incident, who runs the call, who makes sure the same failure does not surprise you a second time. That is where the next lesson, Incident response & learning, picks up.

Progressive deliveryIncident response & learning

Global infrastructureAWS Solutions Architect Associate Microsoft Entra IDAzure Administrator Associate Azure Pipelines & CIAzure DevOps Engineer Professional