You disable the leaked access key for a compromised instance role and move on. Fifteen minutes later the attacker is still making API calls with those credentials. Why?

STS session tokens already issued from the role stay valid until they expire; disabling the key only stops new ones from being minted. Cut the live sessions with a token-issue-time Deny on the role, not just a key change.

A GuardDuty alert says an instance's IAM (Identity and Access Management) role credentials are being used from an external IP address. Why does the lesson tell you to quarantine the instance rather than terminate it right away?

Terminating destroys the two most valuable pieces of evidence — what is held in memory and often the disk — so you quarantine instead: move it to a deny-all security group and detach it from the load balancer and Auto Scaling group, stopping the harm while the instance keeps running and memory stays intact. Quarantine seals the room without burning it down, preserving live memory and the disk for the snapshot that follows.

After containing the instance you replay CloudTrail and find that two minutes after the alert the compromised role called CreateAccessKey, and app-deploy — a user only your deploy pipeline uses and no human signs in as — now has a second active access key stamped with that exact time. What is this, and what do you do?

It is persistence — a spare credential the attacker planted as a way back in — so you delete that key, rotate every secret the role could read, and rebuild the instance from a known-good image. Containment is not clean until you hunt down footholds like this second key, because it outlives any cleanup of the instance itself.

Courses AWS security engineeringIncident response & forensics

Incident response & forensics

Contain, revoke sessions, preserve, hunt persistence.

Expert35 min · lesson 15 of 15

A GuardDuty alert fires at two in the morning. GuardDuty is the AWS service that watches your account for activity that looks like an attack, and right now it's telling you that one of your web servers is calling S3 (Amazon's Simple Storage Service, basically cloud file storage) from an IP address in a datacenter in the Netherlands. The call is being made with the server's own identity. Every AWS server can be handed a bundle of permissions called an IAM role (IAM, short for Identity and Access Management, is the part of AWS that decides who's allowed to do what), like a work badge that opens certain doors, and this role's credentials are the ones talking to that address. Your servers run in Ireland and have never once talked to it. Something has copied the badge and is using it from the outside. What you do in the next ten minutes decides whether you get to understand this attack or spend next week guessing at it. The discipline that saves you is the one a crime-scene investigator uses. Seal off the room. Photograph everything before you touch it. Change the locks. In that order.

Seal the room before you clear it

The reflex, when a machine is compromised, is to kill it. Terminate the instance and make the bad thing stop. Resist that. An EC2 instance is just a running computer that AWS rents you (EC2 stands for Elastic Compute Cloud), and the two most useful pieces of evidence live exactly where terminating destroys them: what's held in memory right now (the attacker's live process and whatever keys or tools it's using) and, often, the disk itself. Terminate first and you've burned down the crime scene to stop a burglar you could have simply locked in a room.

Quarantine is the locking-in. A security group is the firewall wrapped around an instance, the bouncer at the door deciding which traffic gets in and out. Move the instance onto a quarantine security group that allows nothing, and the live session can't phone home to the attacker's server and can't reach your other machines. Detach it from the load balancer and Auto Scaling group too, so it stops serving real users and Auto Scaling doesn't quietly replace it while you work. The instance keeps running. Memory stays intact. You've stopped the bleeding without touching the body.

Before you move, read what tripped the alarm. A GuardDuty finding spells out which credential is being abused and where the calls are coming from.

read the finding that started it all

aws guardduty get-findings --detector-id 12abc34d567e8f901234567890abcdef \
  --finding-ids 7cb9e5a2f1084b3c9d6e2f8a41b7c0d5 --query 'Findings[0]'
{
    "AccountId": "123456789012",
    "Region": "eu-west-1",
    "Type": "UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration.OutsideAWS",
    "Severity": 8,
    "Title": "Credentials for role app-role are being used from an external IP.",
    "Resource": {
        "ResourceType": "AccessKey",
        "AccessKeyDetails": {
            "AccessKeyId": "ASIA5EXAMPLE7QVMPL2C",
            "PrincipalId": "AROA5EXAMPLEID:i-0abc1234",
            "UserName": "app-role",
            "UserType": "AssumedRole"
        }
    },
    "Service": {
        "Action": {
            "ActionType": "AWS_API_CALL",
            "AwsApiCallAction": {
                "Api": "ListBuckets",
                "ServiceName": "s3.amazonaws.com",
                "RemoteIpDetails": {
                    "IpAddressV4": "203.0.113.77",
                    "Country": { "CountryName": "Netherlands" },
                    "Organization": { "Asn": "60068", "OrgName": "Datacamp Limited" }
                }
            }
        },
        "Count": 14,
        "EventFirstSeen": "2026-07-16T01:58:11Z",
        "EventLastSeen": "2026-07-16T02:11:44Z"
    }
}

Read it like an incident report. The role app-role, borrowed by instance i-0abc1234, is being called from 203.0.113.77 in the Netherlands, fourteen times in thirteen minutes, listing your S3 buckets. Those credentials are real and they're live. Now quarantine the instance and take a snapshot. A snapshot is a photograph of the disk copied to storage the attacker can't reach; create-snapshot freezes the volume's exact state for forensics while the original disk keeps running underneath the quarantined instance.

quarantine, verify, then snapshot the volume

aws ec2 modify-instance-attribute --instance-id i-0abc1234 --groups sg-0quar4nt1ne
 
aws ec2 describe-instances --instance-ids i-0abc1234 \
  --query 'Reservations[].Instances[].SecurityGroups'
[
    [ { "GroupName": "quarantine-deny-all", "GroupId": "sg-0quar4nt1ne" } ]
]
 
aws ec2 create-snapshot --volume-id vol-0abc1234 \
  --description "IR-8891 root volume evidence" \
  --tag-specifications 'ResourceType=snapshot,Tags=[{Key=case,Value=IR-8891}]'
{
    "SnapshotId": "snap-0e1f2a3b4c5d6e7f8",
    "VolumeId": "vol-0abc1234",
    "State": "pending",
    "StartTime": "2026-07-16T02:14:37+00:00",
    "VolumeSize": 30,
    "Encrypted": true
}

Turning off the key doesn't end the session

Here's the mistake almost everyone makes on their first incident. You found the leaked credentials, so you disable them, and it feels finished. It isn't. When an instance uses its role, AWS doesn't hand it a permanent password. It issues a short-lived credential called an STS session token (STS, the Security Token Service, is the part of AWS that mints these temporary keys), stamped with the time it was minted and an expiry a few hours out. Disabling the underlying access key stops AWS from minting new tokens, but every token already handed out keeps working until it expires. It's like changing the lock on the front door while the burglar is standing in your hallway holding a day-pass you printed an hour ago. New passes won't print. The one in his hand still opens doors.

To actually cut the live session, you attach a deny policy to the role that rejects any credential issued before right now. This works because of how AWS decides every request. Each API call is checked against all the policies that apply, and an explicit Deny always wins, re-evaluated fresh on every single call. So the attacker's next request, made with a token minted before your cutoff time, hits the Deny and fails on the spot. The day-pass stops working mid-hallway. Attach the policy, then prove it landed: simulate-principal-policy runs a made-up request through the exact same evaluation engine without making the real call, and tells you the decision. You want to see explicitDeny.

deny stale sessions, then verify the decision

aws iam put-role-policy --role-name app-role --policy-name revoke-stale-sessions \
  --policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Deny","Action":"*","Resource":"*","Condition":{"DateLessThan":{"aws:TokenIssueTime":"2026-07-16T02:15:00Z"}}}]}'
 
aws iam simulate-principal-policy \
  --policy-source-arn arn:aws:iam::123456789012:role/app-role \
  --action-names s3:ListAllMyBuckets \
  --context-entries ContextKeyName=aws:TokenIssueTime,ContextKeyType=date,ContextKeyValues=2026-07-16T01:58:00Z \
  --query 'EvaluationResults[0].{action:EvalActionName,decision:EvalDecision,matchedBy:MatchedStatements[0].SourcePolicyId}'
{
    "action": "s3:ListAllMyBuckets",
    "decision": "explicitDeny",
    "matchedBy": "revoke-stale-sessions"
}

That deny policy hits your healthy instances too

The token-issue-time Deny applies to every session using that role, not just the compromised one. If app-role is shared across a fleet of web servers, you've just cut credentials for all of them, and the healthy ones will start failing their S3 and database calls within minutes as their tokens roll over into the Deny. Know your blast radius before you attach it. In a live incident that trade is usually worth it, but tell the service owner it's coming, and design ahead of time so sensitive workloads each get their own dedicated role. Then you can revoke one compromised instance without taking down the fleet.

Find the second way in

Contained isn't clean. A competent attacker, once inside, plants a spare key before you notice them, so that closing the obvious door changes nothing. This is called persistence, and hunting for it is the step people skip and then regret an hour later. The burglar propped a back window open on his way through. Go find it before you declare the house secure. CloudTrail is the account's flight recorder, a log of every API call anyone made, with the time and the source IP. Replay what the compromised identity did from the moment the alert fired, and look for the usual footholds: a brand-new IAM user, an extra access key bolted onto an existing user, a role trust policy quietly edited to trust an outside account, resources launched in a region you never use, or logging switched off. Any one of them is a way back in that outlives you cleaning up the instance.

replay the attacker's IAM calls from CloudTrail

aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=EventName,AttributeValue=CreateAccessKey \
  --start-time 2026-07-16T01:55:00Z --query 'Events[].{caller:Username,when:EventTime}'
[
    { "caller": "app-role/i-0abc1234", "when": "2026-07-16T02:03:52+00:00" }
]
 
aws iam list-access-keys --user-name app-deploy \
  --query 'AccessKeyMetadata[].{id:AccessKeyId,created:CreateDate,status:Status}'
[
    { "id": "AKIA5EXAMPLE1CICD01", "created": "2024-11-02T09:00:00+00:00", "status": "Active" },
    { "id": "AKIA5EXAMPLE9BACKD00", "created": "2026-07-16T02:03:52+00:00", "status": "Active" }
]

And there it is. Two minutes after the alert, the compromised role created an access key for app-deploy, a user only your deploy pipeline uses and nobody signs in as by hand. That second key, minted at 02:03, is the spare. Delete it, rotate every secret the role could read, rebuild the instance from a known-good image instead of scrubbing it in place, and reconnect only once the snapshot is safely stored.

The fork that decides your incident

Compromised instance, credentials live

GuardDuty: instance role used from outside AWS

Terminate it

Evidence destroyed

memory and disk gone, root cause unknowable

Just disable the key

Attacker keeps working

issued STS tokens stay valid until they expire

Quarantine, snapshot, deny old sessions

Contained and preserved

the bleeding stops, the evidence survives

Both obvious moves fail: one burns the evidence, the other leaves the door wide open. The runbook path is the only one that does neither.

Try this

Work through “Find the second way in” yourself on a sandbox you can throw away, following the commands above in order. Then break one step deliberately and re-run, so you have seen the failure before it finds you.

Takeaway

The trap worth remembering here: that deny policy hits your healthy instances too. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Org trails & immutable loggingFinish · self-test

IAM mental modelAWS security for DevOps engineers Multi-account & org identity architectureAdvanced cloud security Resource hierarchy & org policyGCP security Entra ID, Conditional Access & PIMAzure security