CodeDeploy & rollback
Targets, appspec hooks, blue/green, auto-rollback.
Watch a control tower work. Every plane gets cleared, watched on approach, and waved off if something looks wrong before the wheels touch the tarmac. A plain file copy skips all of that. It answers one question, did the new bytes land, and stops there. A release has to answer three more. Is the new version healthy? How much traffic should reach it while you find out? And how do you get back to safety when the answer is no? AWS CodeDeploy (the Amazon Web Services managed deployment service) automates all four. It takes one specific *revision* of your application, pushes it onto compute targets in a sequence you control, runs your health checks along the way, and can reverse course on its own the moment a signal goes bad.
Four nouns carry the whole model. A revision is the exact versioned artifact being deployed: one build, pinned. A deployment group is the named set of targets plus the rules that apply to them. A deployment configuration is the pace, meaning how fast and to how many targets at once. The appspec (application specification file) is the manifest packed inside each revision, telling the agent what to place where and which scripts to run. Get those four straight and every CodeDeploy behavior falls out of them.
Where a revision lands, and what the appspec controls
CodeDeploy runs on three compute platforms, and the word *revision* means something different on each. On EC2/on-premises (Elastic Compute Cloud, the virtual machines you rent, plus your own servers; AWS calls this the *Server* platform) a revision is a bundle: your code, an appspec.yml, and hook scripts, stored in S3 (Simple Storage Service, the object store) or GitHub. The CodeDeploy agent on each instance pulls that bundle and lays the files onto disk. On ECS (Elastic Container Service) and Lambda (the run-a-function-on-demand service) nothing is copied at all. The revision is a small appspec.yaml pointing at a new task definition or a new function version, and CodeDeploy moves traffic to it. That one difference explains most of what follows. File-based targets are a house move: slow, scripted, box by box. Traffic-based targets flip like a light switch.
The appspec earns its keep through lifecycle hooks, named moments where CodeDeploy runs your scripts and holds the whole deployment hostage to their exit code. EC2 gets a long sequence: ApplicationStop, BeforeInstall, AfterInstall, ApplicationStart, ValidateService, plus blue/green traffic hooks like BeforeAllowTraffic and AfterAllowTraffic. ECS gets a shorter, traffic-centric list (BeforeInstall, AfterInstall, AfterAllowTestTraffic, BeforeAllowTraffic, AfterAllowTraffic), and Lambda gets only BeforeAllowTraffic and AfterAllowTraffic. On those two platforms the hooks name Lambda functions rather than shell scripts. A non-zero exit code, or a reported Failed from an ECS or Lambda validation function, fails that lifecycle event and, when rollback is configured, reverses the release. That is how you bake a smoke test *into* the deploy instead of bolting one on afterward.
version: 0.0os: linuxfiles:- source: /destination: /var/www/apphooks:ApplicationStop:- location: scripts/stop_server.shtimeout: 60runas: rootAfterInstall:- location: scripts/install_deps.shtimeout: 300runas: rootApplicationStart:- location: scripts/start_server.shtimeout: 60ValidateService:- location: scripts/smoke_test.sh # non-zero exit => event fails => rollbacktimeout: 120
Deployment configurations: the dial between speed and safety
A deployment configuration is a named policy for pace, nothing else. On EC2 the built-ins are CodeDeployDefault.AllAtOnce (fastest, zero safety margin), HalfAtATime, and OneAtATime (slowest, smallest blast radius). You can also write a custom config with a minimum-healthy-hosts floor, expressed as a count or a percentage, so CodeDeploy never drops the fleet below the capacity you need to keep serving. ECS and Lambda use traffic-shifting configs instead. A *canary* such as ECSCanary10Percent5Minutes sends one fixed slice, waits a fixed bake interval, then sends the rest. A *linear* config such as LambdaLinear10PercentEvery1Minute moves equal increments on a fixed clock. *All-at-once* cuts over immediately. The trade is direct: a longer canary bake gives you more time to catch a regression before everyone sees it, but it slows the release, and under blue/green it stretches the window where you are paying for two fleets at once.
aws deploy get-deployment-config \--deployment-config-name CodeDeployDefault.ECSCanary10Percent5Minutes# {# "deploymentConfigInfo": {# "deploymentConfigName": "CodeDeployDefault.ECSCanary10Percent5Minutes",# "computePlatform": "ECS",# "trafficRoutingConfig": {# "type": "TimeBasedCanary",# "timeBasedCanary": {# "canaryPercentage": 10,# "canaryInterval": 5# }# }# }# }
Blue/green, and what a rollback actually does
In-place deployment updates the machines you already have, like renovating a shop with customers still inside. Blue/green builds a second shop next door, walks the customers over once it is ready, and leaves the old one standing for a while. In AWS terms, blue/green stands up a parallel *green* fleet, shifts traffic to it through a load balancer, and keeps the old *blue* fleet idle for a termination-wait window. ECS blue/green does the same thing with a replacement *task set* behind an ALB's (Application Load Balancer) production and test listeners. Lambda does it by weighting an *alias* between two published versions. Here is the part that quietly sets your recovery time: a CodeDeploy rollback is not an undo. For an EC2 *in-place* group, rolling back means redeploying the last known-good revision as a brand-new deployment. It re-runs every hook and takes a full deploy cycle, minutes rather than seconds. For blue/green, and for ECS and Lambda, the previous version is still running, so rollback is a traffic reroute back to it and finishes in seconds. If fast recovery is what you need, the strategy you picked decided it long before any alarm fired.
version: 0.0Resources:- TargetService:Type: AWS::ECS::ServiceProperties:TaskDefinition: "arn:aws:ecs:us-east-1:111122223333:task-definition/web:42"LoadBalancerInfo:ContainerName: "web"ContainerPort: 8080Hooks:# runs against the TEST listener before any production traffic shifts- AfterAllowTestTraffic: "arn:aws:lambda:us-east-1:111122223333:function:smoke-test"- BeforeAllowTraffic: "arn:aws:lambda:us-east-1:111122223333:function:warm-cache"
Automatic rollback and CloudWatch alarms
Auto-rollback is configured per deployment group, and you choose which events trigger it: DEPLOYMENT_FAILURE, DEPLOYMENT_STOP_ON_ALARM, and DEPLOYMENT_STOP_ON_REQUEST. The trap that catches teams is assuming DEPLOYMENT_FAILURE covers everything. It does not. It fires only when an instance or a lifecycle hook fails. A revision that installs cleanly, passes every hook, and *then* doubles your 5xx rate (server-error responses) or your p99 latency (how slow the worst one percent of requests are) sails straight through, because nothing in the deployment itself broke. Catching that kind of regression takes one or more CloudWatch alarms (CloudWatch is the AWS metrics and alerting service) attached to the deployment group, with DEPLOYMENT_STOP_ON_ALARM enabled. When an alarm crosses into ALARM state mid-deploy, CodeDeploy stops and reverses. Run both layers. Hooks such as ValidateService and AfterAllowTestTraffic are a fast gate on exit codes inside the deploy window. Alarms are the production signal, watching what real users are getting.
# EC2 in-place group: alarm + auto-rollback on failure OR alarmaws deploy create-deployment-group \--application-name web \--deployment-group-name prod \--service-role-arn arn:aws:iam::111122223333:role/CodeDeployRole \--deployment-config-name CodeDeployDefault.HalfAtATime \--ec2-tag-filters Key=Env,Value=prod,Type=KEY_AND_VALUE \--auto-rollback-configuration enabled=true,events=DEPLOYMENT_FAILURE,DEPLOYMENT_STOP_ON_ALARM \--alarm-configuration enabled=true,alarms=[{name=web-5xx-high}]# { "deploymentGroupId": "b1a2c3d4-5e6f-7a8b-9c0d-1e2f3a4b5c6d" }# start a release from a revision in S3aws deploy create-deployment \--application-name web \--deployment-group-name prod \--s3-location bucket=my-artifacts,key=web/build-142.zip,bundleType=zip \--file-exists-behavior OVERWRITE \--description "release 142"# { "deploymentId": "d-A1B2C3D4E" }
aws deploy get-deployment --deployment-id d-A1B2C3D4E \--query 'deploymentInfo.{status:status,rollback:rollbackInfo}'# {# "status": "Stopped",# "rollback": {# "rollbackDeploymentId": "d-F5G6H7I8J",# "rollbackTriggeringDeploymentId": "d-A1B2C3D4E",# "rollbackMessage": "Deployment d-A1B2C3D4E was stopped and rolled back because alarm web-5xx-high went into ALARM state."# }# }# note: the rollback ran as d-F5G6H7I8J, a brand-new deployment — not an undo of d-A1B2C3D4E
Hooks are what separates 'the files are on the box' from 'the service actually works'. Write ValidateService so it fails the deployment when the process is running but not yet ready to take requests. Leave hooks out and CodeDeploy is an expensive scp.
A deployment configuration is where blast radius gets written down: AllAtOnce, HalfAtATime, OneAtATime, canary, linear. Pair the config with CloudWatch alarms so a spike in error rate reverses the release on its own, before anyone gets paged.
Blue/green keeps the old environment alive through cutover, which is why rollback there is a traffic shift rather than a 2 a.m. attempt to rebuild the previous AMI (Amazon Machine Image, the disk snapshot an instance boots from) from memory.
Try this
Read back a deployment group's pace, style, and alarms, then look at its most recent deployments. Run this against a lab application, never production.
aws deploy get-deployment-group --application-name app --deployment-group-name prod \--query 'deploymentGroupInfo.{Config:deploymentConfigName,Style:deploymentStyle.deploymentType,Alarms:alarmConfiguration}' --output jsonaws deploy list-deployments --application-name app --deployment-group-name prod --max-items 3 --output tableaws deploy get-deployment --deployment-id d-EXAMPLE \--query 'deploymentInfo.{Status:status,Config:deploymentConfigName,Error:errorInformation}' --output table
{"Config": "CodeDeployDefault.ECSLinear10PercentEvery1Minutes","Style": "BLUE_GREEN","Alarms": {"enabled": true, "alarms": [{"name": "app-5xx"}]}}d-EXAMPLESucceeded | CodeDeployDefault.ECSLinear10PercentEvery1Minutes | None
Takeaway
CodeDeploy owns three things: the pace, the hooks, and the rollback. Wire the alarms as well, or all you have automated is the push and not the recovery.
Next step in the lab: attach one CloudWatch alarm to a deployment group, break a health check on purpose, and watch the auto-rollback fire.
ApplicationStop hook (and the blue/green BeforeBlockTraffic and AfterBlockTraffic hooks) from the *last successfully deployed* revision rather than the one you are pushing. It has to, because that script runs before the new bundle has even been downloaded. One consequence: ApplicationStop never runs on the very first deployment to an instance, since no earlier appspec exists on that box yet. The nasty part is that a broken stop script which already shipped can hang or fail every future deployment, and a rollback runs the same broken script all over again. Test teardown scripts before they land, keep them boringly simple, guard every command, and exit 0 when the app is already stopped.Production: cost, quotas, and hardening
CodeDeploy itself is free when the targets are EC2, ECS, or Lambda. On-premises instances cost $0.02 per instance update. The real money is indirect: blue/green runs two fleets at the same time, so a long canary bake or a long termination-wait window means double compute for that whole period, plus ALB hours and cross-AZ (Availability Zone, one isolated datacenter group inside a Region) data charges. On limits, the per-Region defaults (most of them adjustable through Service Quotas) sit around 1,000 applications per account, 1,000 deployment groups per application, and 1,000 instances in a single deployment, or 2,000 in us-east-1. A deployment group runs one deployment at a time, so a new release cannot start while another is in progress. A lifecycle hook script defaults to a 3,600-second timeout, which is also the hard ceiling for any single lifecycle event. Harden the setup with a least-privilege service role (the IAM role, short for Identity and Access Management, that CodeDeploy itself assumes), tight EC2 tag filters so one mistyped tag cannot sweep a revision into prod, encrypted S3 artifacts, and for high-risk releases a DeploymentReadyOption of STOP_DEPLOYMENT so the final cutover waits for a human to say go. Those deployment groups, roles, and alarms belong in version control themselves, which is exactly where the next lesson on CloudFormation starts.