Auto Scaling & stateless design

Elastic, self-healing, horizontally-scaled tiers.

A coffee shop that could summon a fully trained barista in two minutes whenever the line got long, then send them home the second it cleared, and pay only for the minutes they actually worked, would never overstaff and never keep you waiting. That is elasticity: capacity that follows demand on its own, in both directions. On AWS the machinery behind it is the EC2 Auto Scaling group, written ASG almost everywhere (EC2 is Elastic Compute Cloud, Amazon's rented virtual machines). Treat it as a shift supervisor for a fleet of identical servers. It keeps the headcount right, replaces any server that dies, and spreads the fleet across separate data centres. Put an Application Load Balancer (ALB, the traffic cop that shares incoming requests out across servers) in front of it and you have the classic self-healing web tier. Below you build one from the command line, break it on purpose, and watch it put itself back together.

What an Auto Scaling group is made of

Two pieces do all the work. A launch template is the recipe card: which AMI (Amazon Machine Image, the frozen disk image a server boots from), what size of instance, which security groups (the firewall rules around it), and the startup script that installs your app. The ASG is the supervisor that stamps out copies of that recipe. Three numbers govern it. Min is the floor you never drop below, even at 3 a.m. Max is the ceiling, and it doubles as a spending circuit breaker. Desired is the current setpoint, the number that scaling policies move up and down between the other two. Spreading across Availability Zones (AZs, physically separate data centres inside one region) is a subnet list and nothing more: hand --vpc-zone-identifier subnets in three different AZs and the group balances instances across them. The flag that quietly decides everything is --health-check-type. Its default, EC2, trusts the status checks reported by the hypervisor (the host software your virtual machine runs on top of), so a server counts as healthy whenever the machine is running and answering on the network, even if the app inside it crashed an hour ago. Set it to ELB and the verdict moves to the load balancer's HTTP health checks, which are real web requests to a real URL on the server, testing the thing your users actually touch. The built-in EC2 checks stay switched on underneath, and you cannot turn those off. For anything sitting behind a load balancer, use ELB. The exam asks about this difference constantly.

# 1. The recipe: a launch template (AMI, size, SG, bootstrap)
aws ec2 create-launch-template \
  --launch-template-name web-tier \
  --launch-template-data '{
    "ImageId": "ami-0abcdef1234567890",
    "InstanceType": "t3.micro",
    "SecurityGroupIds": ["sg-0f1e2d3c4b5a69788"]
  }'
 
# 2. The supervisor: an ASG across three AZs, wired to an ALB target group
aws autoscaling create-auto-scaling-group \
  --auto-scaling-group-name web-asg \
  --launch-template "LaunchTemplateName=web-tier,Version=\$Latest" \
  --min-size 2 --max-size 12 --desired-capacity 4 \
  --vpc-zone-identifier "subnet-0aa1b2c3d4e5f6789,subnet-0bb2c3d4e5f67890a,subnet-0cc3d4e5f67890ab1" \
  --target-group-arns arn:aws:elasticloadbalancing:us-east-1:123456789012:targetgroup/web-tg/73e2d6bc24d8a067 \
  --health-check-type ELB --health-check-grace-period 120
 
# 3. Verify (step 2 printed nothing — that's success)
aws autoscaling describe-auto-scaling-groups \
  --auto-scaling-group-names web-asg \
  --query 'AutoScalingGroups[0].{Min:MinSize,Desired:DesiredCapacity,Max:MaxSize,HealthCheck:HealthCheckType,AZs:AvailabilityZones}'
{
    "Min": 2,
    "Desired": 4,
    "Max": 12,
    "HealthCheck": "ELB",
    "AZs": ["us-east-1a", "us-east-1b", "us-east-1c"]
}

create-launch-template prints the new template's metadata straight back at you. create-auto-scaling-group prints nothing at all when it works, so silence plus a zero exit code is the success message. That is exactly why the verifying describe call is worth turning into a reflex. A few minutes later you have four instances spread 2-1-1 across the three zones, registered with the load balancer, and judged on HTTP health checks once the 120-second grace period runs out.

The policies that move the desired number

Four families of policy move desired between min and max. There is a fifth, *simple scaling*, the legacy ancestor of step scaling; know the name for the exam and never pick it for real work. Target tracking behaves like a thermostat. You name one metric and one number, say keep average CPU (processor) use near 50 percent, and AWS writes the CloudWatch alarms (CloudWatch is the AWS metrics and alarms service) and does the arithmetic for you. Step scaling is a ladder you build by hand: CPU over 70, add two instances; over 85, add four. Pick it when you want every rung spelled out. Scheduled scaling covers the rhythms you already know about, like the 9 a.m. login stampede. Predictive scaling studies up to two weeks of past metrics, needs at least 24 hours of them before it produces a first forecast, and adds capacity ahead of load that repeats on a cycle. Exam shortcut: when a question asks for scaling with the *least operational overhead*, the answer is nearly always target tracking.

aws autoscaling put-scaling-policy \
  --auto-scaling-group-name web-asg \
  --policy-name keep-cpu-at-50 \
  --policy-type TargetTrackingScaling \
  --target-tracking-configuration '{
    "PredefinedMetricSpecification": { "PredefinedMetricType": "ASGAverageCPUUtilization" },
    "TargetValue": 50.0
  }'
{
    "PolicyARN": "arn:aws:autoscaling:us-east-1:123456789012:scalingPolicy:6d8972f3-efc8-4d61-b4a2-92c4e1f9a3b7:autoScalingGroupName/web-asg:policyName/keep-cpu-at-50",
    "Alarms": [
        {
            "AlarmName": "TargetTracking-web-asg-AlarmHigh-2a9c7d4e-8b13-4f6a-9c05-d1e8f4b72a61",
            "AlarmARN": "arn:aws:cloudwatch:us-east-1:123456789012:alarm:TargetTracking-web-asg-AlarmHigh-2a9c7d4e-8b13-4f6a-9c05-d1e8f4b72a61"
        },
        {
            "AlarmName": "TargetTracking-web-asg-AlarmLow-7f3b1c9a-4e28-4d07-a5b6-c8d92e01f345",
            "AlarmARN": "arn:aws:cloudwatch:us-east-1:123456789012:alarm:TargetTracking-web-asg-AlarmLow-7f3b1c9a-4e28-4d07-a5b6-c8d92e01f345"
        }
    ]
}

Look closely at that response. One policy, but *two* alarms, and they are lopsided on purpose. AlarmHigh fires after 3 straight minutes above the target, because being slow is something your users feel, so you add capacity eagerly. AlarmLow insists on 15 straight minutes below its own threshold, which sits a cushion below the target itself, before it takes anything away. Removing capacity lazily is a deliberate choice: flapping (add, remove, add) costs you more than a handful of idle minutes ever would. Both alarms belong to AWS, not to you. The documentation says plainly not to create, edit or delete them by hand. Change the policy instead. During a scale-out, the *instance warmup* window protects the arithmetic in two directions at once. A server that is still booting has almost no CPU load, and that near-zero reading is left out of the group's average, so it cannot drag the number down and set off a pointless scale-in. At the same time, that booting server still counts toward capacity for the next scale-out decision, so the policy never orders more servers to replace ones already on their way. If a policy sets no warmup of its own, it falls back to the group's *default instance warmup*, and if that is missing too, to the default cooldown of 300 seconds.

Load that follows a clock you already own has no need of a thermostat. Warm the capacity up on a schedule instead, and set it in a real time zone name from the IANA database (the standard list, with entries like America/New_York) so daylight saving shifts come along for free:

# 07:30 every weekday (five cron fields; 1-5 = Monday-Friday)
aws autoscaling put-scheduled-update-group-action \
  --auto-scaling-group-name web-asg \
  --scheduled-action-name weekday-morning-preload \
  --recurrence "30 7 * * 1-5" \
  --time-zone "America/New_York" \
  --min-size 4 --desired-capacity 8 --max-size 20
 
# Confirm it registered
aws autoscaling describe-scheduled-actions \
  --auto-scaling-group-name web-asg \
  --query 'ScheduledUpdateGroupActions[].{Name:ScheduledActionName,Cron:Recurrence,Desired:DesiredCapacity}'
[
    {
        "Name": "weekday-morning-preload",
        "Cron": "30 7 * * 1-5",
        "Desired": 8
    }
]

Break it, watch it heal

Self-healing is the same machinery aimed at failure instead of at load. Any instance that flunks its health checks gets terminated and replaced, with nobody paged. You can set the whole loop off safely by telling the group an instance is sick yourself:

# Simulate a failure: mark one instance unhealthy
aws autoscaling set-instance-health \
  --instance-id i-0c3f7e2a9d8b1e654 \
  --health-status Unhealthy
 
# a minute or two later, read the audit trail
aws autoscaling describe-scaling-activities \
  --auto-scaling-group-name web-asg --max-items 2 \
  --query 'Activities[].{What:Description,Status:StatusCode}'
[
    {
        "What": "Launching a new EC2 instance: i-0a9b8c7d6e5f41230",
        "Status": "Successful"
    },
    {
        "What": "Terminating EC2 instance: i-0c3f7e2a9d8b1e654",
        "Status": "Successful"
    }
]

That activity log is your audit trail. Every entry also carries a Cause field spelling out which health check failed and when, and it is the first thing to read any time capacity behaves oddly. Notice the ordering you do *not* get by default: the group terminates first and launches second, so for a short window it runs below the desired count. (An *instance maintenance policy* flips that around to launch-before-terminate, which matters for small groups that cannot afford the dip.) Meanwhile the load balancer stops sending traffic to a genuinely sick instance the moment it trips the target group's unhealthy threshold, two failed checks in a row by default, long before the replacement has finished booting.

Statelessness: the deal that makes scaling safe

All of this rests on one property. Your app tier has to be stateless, meaning no single server holds anything that would hurt to lose the instant it disappears. Scale-in never asks permission, so every scaling event is a small outage for whatever lived only on that machine's disk or in its memory. Session data therefore belongs in ElastiCache (managed Redis, reads in well under a millisecond) or DynamoDB (a serverless key-value database with no cluster for you to size). Uploaded files go to S3 (Simple Storage Service). Logs get shipped off the box to CloudWatch. Once any server can answer any request, servers become interchangeable: cattle, not named pets. The tempting shortcut here is ALB *sticky sessions*, a cookie that pins each user to one server. Treat it as a crutch. It piles load unevenly onto whichever instances got popular, and every scale-in logs out everyone pinned to the machine that vanished. The one thing stickiness cannot smooth, the target group's *deregistration delay* does: it keeps draining in-flight requests for 300 seconds by default, tunable anywhere from 0 to 3600, before a departing instance is cut off.

Trade-offs, and what the exam likes to ask

Adding more instances, horizontal scaling, is the cloud default. Vertical scaling, swapping in one bigger machine, needs a stop and a start, hits a hard ceiling at the largest instance size, and leaves you a single failure away from having nothing at all. Keep it as a last resort for workloads that refuse to spread out. Databases play by different rules, read replicas and caches for reads, partition design for writes, and the RDS, Aurora & DynamoDB lesson covers those (RDS is Relational Database Service). Your job *here* is keeping state out of the app tier so it can grow and shrink freely. Three exam details are worth memorising. Launch configurations are the frozen predecessor of launch templates: no new features, none coming, and the wrong answer on any design built today. The default termination policy picks the Availability Zone holding the most instances, then inside that zone kills the instance with the most out-of-date configuration, launch-configuration instances first, then the oldest launch template version, so it quietly rebalances zones and rolls out fixes as a side effect. And for apps that boot slowly, warm pools park pre-initialised instances in a stopped-but-ready state, cutting scale-out time from minutes to seconds for a few cents.

Everything so far scaled on how hard your servers are *working*. Scaling on how much work is *waiting* is cleaner still. An Auto Scaling group can target the backlog per instance in an SQS queue (Simple Queue Service, the AWS managed message queue), which lets the code producing work and the code consuming it grow and shrink independently, without either side ever meeting the other. That queue, and the loosely coupled architectures it makes possible, is the next lesson: SQS, SNS & EventBridge.

Desired capacity is a wish. Min and max are the guardrails that keep the wish sane. For most web tiers, target tracking on ALBRequestCountPerTarget or on CPU beats a step policy you tuned by hand. Cooldowns exist so your fleet does not thrash.

Try this

Read back an Auto Scaling group's min, desired and max, plus the target tracking policy attached to it. Both commands only read, so point them at a lab group and nothing changes.

aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names web-asg \
  --query 'AutoScalingGroups[0].{Min:MinSize,Desired:DesiredCapacity,Max:MaxSize,AZs:AvailabilityZones}' --output table
aws autoscaling describe-policies --auto-scaling-group-name web-asg \
  --query 'ScalingPolicies[].{Name:PolicyName,Type:PolicyType,Metric:TargetTrackingConfiguration.PredefinedMetricSpecification.PredefinedMetricType}' --output table

-----------------------------------------------
|         DescribeAutoScalingGroups           |
+----------+---------+-----+------------------+
| Desired  |   Max   | Min | AZs              |
+----------+---------+-----+------------------+
|  2       |  8      |  2  | us-east-1a,1b    |
+----------+---------+-----+------------------+
cpu-tt | TargetTrackingScaling | ASGAverageCPUUtilization

Takeaway

Remember: scale identical, disposable servers, and keep sessions and anything durable off the box, so replacing a node is a boring non-event.

Next: break one lab instance on purpose and confirm that the ASG and its ALB health check replace it without anyone filing a ticket.

Related