Evidence acquisition

Snapshots, memory, integrity, chain of custody.

A crime-scene investigator turns up with gloves, tamper-evident bags, and a camera, because the moment you touch evidence you change it. A cloud intrusion is the same job in a room that is evaporating while you stand in it. RAM (random access memory, the chips holding everything the machine is thinking right now) empties the instant an instance stops. Instance-store disks vanish at termination. An attacker holding admin credentials can delete snapshots faster than you can create them. Evidence acquisition is the craft of getting complete, provably unaltered copies of memory, disk, and logs out of that room without disturbing the originals. A detection firing says something looks wrong; it does not say an incident happened. Evidence is what settles the difference, which is why acquisition starts before anyone has confirmed anything. The previous lesson gave you the DFIR (digital forensics and incident response) *process*. This one is the hands-on-keyboard half: real commands, run in the order that volatility dictates.

The order of volatility

Evidence spoils at different speeds, the way a fridge does when the power cuts out: the ice cream goes first, the tins keep for years. Order of volatility ranks evidence by how fast it disappears and tells you to collect the most perishable first. It is written down in RFC 3227 (Request for Comments, the internet's standards-and-guidance document series) and it is a reliable exam question on GCFA (GIAC Certified Forensic Analyst) and CySA+ (CompTIA Cybersecurity Analyst). In the cloud the shelf lives run like this. RAM goes at stop or reboot. Instance-store volumes, the temporary disks physically bolted to the host, go at stop or termination. EBS (Elastic Block Store, the network-attached disk service) volumes survive, but every second of runtime rewrites parts of them. Logs already shipped to central immutable storage keep longest; if you did the work in *Log pipelines*, the audit trail is the one thing the attacker cannot burn. Here is the cloud twist. Ordinary containment reflexes destroy evidence: stop the instance and you wipe its RAM, terminate it and the ephemeral disks are released. So the first acquisition action is to freeze the box where it stands. Switch on termination protection, then swap its security group for a quarantine group that drops everything except outbound port 443 to the SSM (AWS Systems Manager) VPC (virtual private cloud) endpoints. Close that last hole as well and your remote memory-capture channel goes deaf.

# 1. Nobody terminates this box while it holds evidence
aws ec2 modify-instance-attribute \
  --instance-id i-0f3e9d2c1b7a8e4f5 --disable-api-termination
 
# Auto Scaling IGNORES that flag -- if the box is in an ASG,
# also protect it from scale-in:
aws autoscaling set-instance-protection \
  --instance-ids i-0f3e9d2c1b7a8e4f5 \
  --auto-scaling-group-name web-asg --protected-from-scale-in
 
# 2. Quarantine SG: no inbound, outbound 443 only to the SSM VPC
#    endpoints + the S3 gateway prefix list (keeps capture alive)
aws ec2 modify-instance-attribute \
  --instance-id i-0f3e9d2c1b7a8e4f5 --groups sg-0aa11bb22cc33dd44
 
# Verify the freeze took:
aws ec2 describe-instance-attribute \
  --instance-id i-0f3e9d2c1b7a8e4f5 --attribute disableApiTermination
# {
#     "InstanceId": "i-0f3e9d2c1b7a8e4f5",
#     "DisableApiTermination": { "Value": true }
# }

Memory first: AVML over SSM

Disk is the filing cabinet. RAM is the desk, covered in whatever the machine is working on this second: fileless malware injected straight into a running process that never touched a file, decrypted C2 (command and control, the attacker's remote-control channel) configuration, TLS (Transport Layer Security, the encryption behind HTTPS) session keys, the live socket table showing who is talking to whom. A memory dump is a byte-for-byte copy of physical RAM, and the de-facto exchange format is LiME (Linux Memory Extractor). LiME began life as a kernel module you had to compile against the exact kernel running on the victim host, which is miserable work in the middle of an incident. AVML (Acquire Volatile Memory for Linux), Microsoft's open-source acquisition tool, sidesteps all of it: one static binary, nothing to compile. It tries /dev/crash, then /proc/kcore, then /dev/mem until one works, and writes LiME-format output. Pass --compress and you get a Snappy-compressed variant that its convert subcommand turns back into plain LiME for analysis. Deliver it through SSM Run Command rather than SSH (Secure Shell, the usual remote-login protocol), because an SSH login creates sessions, auth-log entries, and shell history on the very host you are preserving. Two rules about placement. Stage the binary from your own IR (incident response) bucket, never from the internet mid-incident. Write the dump to a freshly attached evidence volume: not the disk you are about to snapshot, and not /dev/shm, which eats the RAM you are capturing. Know the limitation too. On a big host the capture runs for minutes while pages keep changing underneath it, so a dump is a *smear* of memory rather than a clean freeze-frame. Take it early.

# Stage AVML from YOUR bucket, dump to the attached evidence volume
# (--compress = Snappy page-level compression), then ship it to the vault
aws ssm send-command \
  --instance-ids i-0f3e9d2c1b7a8e4f5 \
  --document-name "AWS-RunShellScript" \
  --parameters 'commands=[
      "aws s3 cp s3://ir-toolkit-555566667777/avml /opt/avml && chmod +x /opt/avml",
      "/opt/avml --compress /mnt/evidence/web01.mem.lime",
      "aws s3 cp /mnt/evidence/web01.mem.lime s3://ir-evidence-vault/IR-2481/"
    ]' \
  --output text --query Command.CommandId
# 7c2f41a9-88a0-4c3e-b1d5-0e6f9a2b3c4d
 
aws ssm get-command-invocation \
  --command-id 7c2f41a9-88a0-4c3e-b1d5-0e6f9a2b3c4d \
  --instance-id i-0f3e9d2c1b7a8e4f5 --query Status
# "Success"
 
# On the ANALYSIS box: convert back to plain LiME, then sanity-check
avml convert web01.mem.lime web01.lime
vol -f web01.lime banners.Banners
# Volatility 3 Framework 2.28.0
# Offset        Banner
# 0x141c001a0   Linux version 6.8.0-1021-aws (buildd@lcy02-amd64-032) ...

Disk: snapshot the volume, analyze a copy

An EBS snapshot is a block-level, point-in-time copy of a volume, stored incrementally in S3 (Simple Storage Service, the AWS object store) behind the scenes. It beats running dd (the classic Unix disk-copying tool) on the suspect host three ways over. Nothing executes on the compromised box, which matters more than it sounds: a rootkit can lie to dd about what is on disk, and it cannot lie to the storage layer underneath. The original volume is never mounted or modified. And the point in time is fixed the instant the API (application programming interface) call is accepted, so a pending state means blocks are still copying, not that the image is drifting. From the snapshot you get two analysis paths. Create a volume from it and attach that read-only to an isolated analysis instance in the same availability zone. Or pull a raw image file with coldsnap, the AWS Labs command-line tool that reads snapshot blocks over the EBS direct APIs with nothing attached at all. A detection-engineering note while you are here: ebs:GetSnapshotBlock is exactly how attackers exfiltrate disks, so the same API you use for acquisition belongs in the watchlists you built in *SIEM & correlation*.

aws ec2 create-snapshot \
  --volume-id vol-0a1b2c3d4e5f67890 \
  --description "IR-2481 web-01 root volume" \
  --tag-specifications 'ResourceType=snapshot,Tags=[{Key=Case,Value=IR-2481},{Key=Hold,Value=legal}]'
# {
#     "SnapshotId": "snap-0e9f8a7b6c5d43210",
#     "State": "pending",
#     "StartTime": "2026-07-13T09:41:22+00:00",
#     "VolumeSize": 100,
#     "Encrypted": true
# }
 
# The point in time is already fixed; wait only because you want to read it
aws ec2 wait snapshot-completed --snapshot-ids snap-0e9f8a7b6c5d43210
 
# Pull a raw image over the EBS direct APIs -- nothing gets attached;
# each block ships with a SHA-256 checksum in the GetSnapshotBlock response
coldsnap download snap-0e9f8a7b6c5d43210 web01-root.img
# 100 GiB raw image (~6 min)
 
# Alternative: mount a copy read-only on your analysis instance
# aws ec2 create-volume --snapshot-id snap-0e9f8a7b6c5d43210 \
#   --availability-zone us-east-1a

Integrity: hash at birth, vault in WORM

A cryptographic hash (use SHA-256, the 256-bit Secure Hash Algorithm) is the tamper-evident seal on a pill bottle. Recompute it later and one flipped bit anywhere in the file gives a completely different value, which is how you *prove* to a court, an insurer, or your own future self that the artifact you analyzed is the artifact you acquired. Hash every artifact the moment it exists, before it moves anywhere. When you image an attached device node instead of using coldsnap, reach for dc3dd, the US Department of Defense Cyber Crime Center's forensically patched build of GNU dd, which hashes inline while it images: dc3dd if=/dev/nvme1n1 of=web01.dd hash=sha256 log=web01.log. Then vault everything in WORM storage (write once, read many). S3 Object Lock in COMPLIANCE mode means nobody deletes the object or shortens its retention before the date passes, and nobody includes the account root user. GOVERNANCE mode is the softer variant that privileged principals can bypass, so it fails the hostile-insider test. Put the vault bucket in a dedicated forensics account with write-only cross-account access, so production admin credentials, yours or stolen, can never reach it. On cost: snapshots run roughly five cents per GB-month, and how long you keep them is a legal decision rather than a storage-budget one.

sha256sum web01-root.img web01.mem.lime | tee IR-2481-hashes.txt
# 5f2ac1b7e8d94063a2c5f7e1908b4d6c3a1f0e9d8c7b6a5f4e3d2c1b0a9f8e7d  web01-root.img
# 9c81f4a2d7e0b3c6598a1f4e7d0c3b6a2958e1d4c7f0a3b6d9e2c5f8a1b4d702  web01.mem.lime
 
# A single S3 PUT caps out at 5 GiB -- a 100 GiB image goes up via the
# multipart-aware high-level command...
aws s3 cp web01-root.img s3://ir-evidence-vault/IR-2481/web01-root.img
# upload: ./web01-root.img to s3://ir-evidence-vault/IR-2481/web01-root.img
 
# ...then gets locked. The bucket was created with
# --object-lock-enabled-for-bucket; COMPLIANCE mode = nobody, root
# included, can delete before the date.
aws s3api put-object-retention \
  --bucket ir-evidence-vault \
  --key IR-2481/web01-root.img \
  --retention 'Mode=COMPLIANCE,RetainUntilDate=2028-07-13T00:00:00Z'

Chain of custody, as code

Chain of custody is the signature sheet taped to the evidence bag: an unbroken written record of who collected each artifact, when, from where, by what method, and every pair of hands it passed through afterwards. It is what turns a folder of files into evidence a court, a regulator, or an HR panel will accept. The cloud quietly does part of the paperwork for you. CloudTrail (the AWS audit log of API calls) already recorded which IAM (Identity and Access Management) principal called CreateSnapshot, from which IP address, at which second, so custody is partly self-documenting. One more reason API-driven acquisition beats laptop-and-USB forensics. You still owe one manifest per artifact, written at acquisition time and vaulted under the same Object Lock retention as the evidence itself:

cat > IR-2481-web01-root.manifest.json <<'EOF'
{
  "case":         "IR-2481",
  "artifact":     "web01-root.img",
  "sha256":       "5f2ac1b7e8d94063a2c5f7e1908b4d6c3a1f0e9d8c7b6a5f4e3d2c1b0a9f8e7d",
  "source":       "vol-0a1b2c3d4e5f67890 on i-0f3e9d2c1b7a8e4f5, us-east-1a",
  "method":       "EBS snapshot snap-0e9f8a7b6c5d43210 -> coldsnap download",
  "collected_by": "arn:aws:sts::555566667777:assumed-role/ir-responder/jchen",
  "collected_at": "2026-07-13T10:14:52Z"
}
EOF
 
aws s3api put-object \
  --bucket ir-evidence-vault \
  --key IR-2481/web01-root.manifest.json \
  --body IR-2481-web01-root.manifest.json \
  --object-lock-mode COMPLIANCE \
  --object-lock-retain-until-date 2028-07-13T00:00:00Z

Run this whole sequence from a script, not from memory. At 3 a.m. during a live incident a checklist beats recall every time, and every call above is what your *SOAR & response* playbooks should be automating. What you hold at the end is a hashed disk image, a verified memory dump, and an immutable log trail: provably intact and completely inert. Bytes on their own do not explain an intrusion. *Sequence* does. The next lesson feeds these exact artifacts to log2timeline and plaso, grinding filesystem metadata, memory artifacts, and log events into one super timeline, where the story of the attack finally becomes readable.

Custody in the cloud is a chain of hashes, ticket IDs, and access logs rather than a plastic bag with a signature on the flap. When you copy a snapshot into the evidence account, record who started the copy, the source and destination snapshot IDs, the SHA-256 of any exported image file, and the storage prefix it landed in. Shape IAM so responders can write evidence and cannot delete it. When legal asks six months later whether that disk could have been altered, you want Object Lock retention and CloudTrail from the evidence account, not a shrug.

Try this

Put a lab volume through the same motions. Snapshot it, copy a small exported artifact into an evidence bucket prefix, hash it, and tag the object, so you finish with a custody mini-trail you could put on screen during a tabletop exercise.

$ VOL=vol-0aa11bb22
$ SNAP=$(aws ec2 create-snapshot --volume-id $VOL --description "IR-lab $(date -u +%Y%m%dT%H%M%SZ)" \
    --query SnapshotId --output text)
$ echo "snapshot=$SNAP"
snapshot=snap-0f00dcafe
$ aws s3 cp ./memory-sample.lime s3://ir-evidence-lab/IR-2026-07-24/memory-sample.lime
$ sha256sum memory-sample.lime
3c1a...  memory-sample.lime
$ aws s3api put-object-tagging --bucket ir-evidence-lab \
    --key IR-2026-07-24/memory-sample.lime \
    --tagging 'TagSet=[{Key=case,Value=IR-2026-07-24},{Key=sha256,Value=3c1a...}]'
# Ticket note: snap-0f00dcafe + s3 object + sha256 + operator + UTC time.

Takeaway

Acquire in volatility order, copy the evidence out, hash it the moment it exists, and take delete away from everyone who touches it. Cloud forensics is custody at API speed, snapshots and memory and logs, with integrity you can still explain a year later in a room full of lawyers.

Next step: stand up an evidence AWS account (or project) with write-once permissions, and rehearse one snapshot-to-evidence copy on a quiet afternoon, long before you need it at 2 a.m.

Related