CoursesSoftware supply chain in depthSupply-chain threat model

Supply-chain threat model

The trust handoffs from commit to deploy, and how they break.

Advanced30 min · lesson 1 of 15

In March 2024 a Microsoft engineer called Andres Freund was poking at a test machine and noticed something petty. Logging in over SSH (secure shell, the standard way to get a remote command line) was taking about half a second longer than it should, and the processor was working hard for no visible reason. Half a second. He pulled on that thread anyway, and out came the xz-utils backdoor: a nearly two-year campaign of patient social engineering that had planted a hidden remote-control implant inside liblzma, a compression library that the sshd login service loads indirectly on most mainstream Linux distributions. It was caught by luck, days before it reached stable releases. That is the shape of every supply-chain attack. Poison one link early and the payload rides downstream to millions of machines with a valid signature already attached. To defend that chain, you first have to map it the way the attacker maps it.

Map the pipeline the way a burglar maps a house

A burglar does not think about your living room. He thinks about the weakest door. Two cheap modelling tools let you think the same way. An attack tree writes the attacker's goal at the top, here it is "ship malicious code that inherits your good name", then branches downward into every route that reaches it. STRIDE is the checklist for labelling each route by threat type: Spoofing (pretending to be someone trusted), Tampering (changing something), Repudiation (denying you did it), Information disclosure (leaking data), Denial of service (breaking availability), and Elevation of privilege (gaining rights you should not have). Lay those over the SDLC (software development lifecycle, the path code takes from a keyboard to production) and five handoffs appear. At each one, somebody accepts somebody else's output on faith: source (the commit), dependencies (the code you pull in), build (compiling and packaging), publish (signing and pushing to a registry), and deploy (letting the workload into a running cluster). Break a single handoff and every link after it trusts the tampered output on autopilot. That is the attacker's whole advantage. They never need your production database. They need one earlier, quieter link, because trust only ever flows downstream.

Source is the first door, and a stolen key is the usual way in. In 2021 attackers found a Google Cloud Storage credential baked into a public Codecov Docker image, lifted it out, and used it to quietly edit Codecov's Bash Uploader. That uploader is a script thousands of CI (continuous integration, the servers that build and test your code automatically) pipelines download and pipe straight into a shell on every run. The edited version copied every environment variable off the runner and sent it to the attacker. CI runners are where the valuables sit: cloud keys, registry tokens, signing material. In STRIDE terms one move gave the attacker Spoofing of a trusted publisher, Tampering with its script, and Information disclosure of every downstream customer's secrets. The fixes are unglamorous. Require two humans on protected branches, sign your commits, and treat any script your build fetches as code you pin and verify by digest instead of downloading fresh each time. Those are the concerns the SLSA (Supply-chain Levels for Software Artifacts, the industry framework this course is built around) Source track covers.

Most of your code was written by strangers

Most of the code you ship, you did not write. A normal service pulls in hundreds of transitive dependencies (the packages your packages depend on, several layers deep), and behind each one sits a maintainer you have never met and cannot vet. Two incidents show exactly how that breaks. In 2018 the popular npm (Node package manager) package event-stream was handed over to a friendly volunteer who offered to take maintenance off the tired owner's hands. That new "maintainer" shipped a small version bump adding one dependency, flatmap-stream, which carried an encrypted payload. The payload decrypted and woke up only inside the build of the Copay bitcoin wallet, where it stole wallet keys, and stayed asleep everywhere else. Six years later xz-utils went further. The attacker spent two years earning co-maintainer status, hid the backdoor as scrambled bytes inside binary files that looked like ordinary test fixtures, then wired it in at build time through a doctored build-to-host.m4 macro (a snippet of build configuration script) that ran only when you built from the released tarball, never from the git tree. Anyone reading the code on GitHub saw clean source. Both are Tampering escalating into Elevation of privilege, and no human skimming a diff was ever going to catch either.

The answer for dependencies is content-addressing, which works like the tamper-evident seal on a medicine bottle. You record the exact cryptographic hash of every package in a lockfile, so if the bytes that arrive differ by so much as one character, the install stops. SLSA does not grade your third-party dependencies directly, but the controls around them do that work: a hash-pinned lockfile, an SBOM (software bill of materials, an itemised list of everything inside what you shipped), and VEX (Vulnerability Exploitability eXchange, a statement of which of those findings actually reach your code) to triage the results. Together they turn "trust the registry" into "verify the bytes". Here is what a poisoned update looks like once the seal is on. It stops being a silent swap and becomes a loud failure:

requirements.txt + install with hash pinning
# requirements.txt — every wheel bound to its content hash
requests==2.31.0 \
--hash=sha256:58cd2187c01e70e6e26505bca751777aa9f2ee0b7f4300988b709f44e013003f
$ pip install --require-hashes -r requirements.txt
Collecting requests==2.31.0
Downloading requests-2.31.0-py3-none-any.whl (62 kB)
ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE.
If you have updated the package versions, please update the hashes. Otherwise,
examine the package contents carefully; someone may have tampered with them.
requests==2.31.0 from https://files.pythonhosted.org/.../requests-2.31.0-py3-none-any.whl:
Expected sha256 58cd2187c01e70e6e26505bca751777aa9f2ee0b7f4300988b709f44e013003f
Got 7f4e6f0b1c2d9e83a5b1d0c4e6f8a2b3c5d7e9f1a2b3c4d5e6f7a8b9c0d1e2f3a

When the factory itself is the attacker

Build is the most valuable link to own, because whatever comes out of it is trusted by definition. SolarWinds in 2020 is the case worth reading slowly. A purpose-built implant called SUNSPOT sat on the build server watching the running process list for MsBuild.exe. The moment it saw the Orion product start compiling, it swapped one source file on disk for a backdoored copy, let the build compile that and Authenticode-sign the result (Authenticode is Microsoft's code-signing scheme), then put the clean file back. Git history: clean. Developer laptops: clean. Later forensics on the source: clean. Meanwhile the shipped DLL (dynamic link library, a compiled component that Windows programs load at runtime) carried the SUNBURST backdoor and a perfectly valid signature, because it had been signed with SolarWinds' own real key. That is the sharpest lesson in the whole field. A signature proves nothing about integrity. What would have caught it is an isolated build on a hardened platform, where one build run cannot reach into another and the signing key stays out of reach of the build steps, emitting provenance that records exactly which sources and which steps produced the artifact. Then a verifier can prove the built bytes came from the commit you actually reviewed. That is SLSA Build Level 3.

Signed does not mean safe
Teams ask "is it signed?", get a yes, and file the artifact under secure. SolarWinds is why that fails. The backdoored DLL carried SolarWinds' own valid signature. A signature ties an artifact to a publisher's identity, and says nothing about whether the build that produced it was interfered with. Verification in production has to pin the signer identity you expect and the builder you expect, then check the provenance against the source commit. Confirming that some signature exists is not a check.

Publish is where an artifact gets a name and an address that other people fetch by reference. Names can be moved. Content cannot. In March 2025 the widely used GitHub Action tj-actions/changed-files was compromised (CVE-2025-30066, where CVE stands for Common Vulnerabilities and Exposures, the public catalogue of known flaws). Armed with a leaked token, the attacker force-pushed a malicious commit and then repointed every version tag, v1 through v45, at it. Every pipeline that referenced the action by a floating tag pulled the payload on its next run. The injected code dumped the CI runner's memory, secrets included, into build logs that anyone could read. A Git tag is a sticky label you can peel off and move somewhere else. A commit SHA (secure hash algorithm digest, calculated from the content itself) or an OCI (Open Container Initiative) image digest is derived from the bytes, so it cannot be repointed. Pin every action and every image by immutable digest, and verify build provenance downstream so a changed source is still caught when some reference moves:

a tag is mutable; a commit digest is not
# A tag is just a movable pointer. Resolve what it points to before you trust it:
$ git ls-remote https://github.com/tj-actions/changed-files refs/tags/v46
ed68ef82c095e0d48ec87eccea555d944a631a4c refs/tags/v46
# During CVE-2025-30066 the attacker force-pushed ONE malicious commit and
# repointed the tags v1..v45 at it: 0e58ed8671d6b60d0890c21b07f8835ace038e67
# Every pipeline pinned to a floating tag pulled that payload on its next run.
#
# Harden: review a specific commit, then pin the action to that immutable
# 40-char SHA — never a tag, and never the attacker's commit above:
# uses: tj-actions/changed-files@ed68ef82c095e0d48ec87eccea555d944a631a4c # v46

Stop trusting the label, start checking the evidence

Every incident above ran on the same default setting. A downstream link trusts upstream output because of where it came from and what it is called. The inversion this course builds is to make each stage produce evidence a machine can check, and to make the deploy gate refuse anything that turns up without it. Provenance is the word to hold onto. It is a receipt for the build: a signed, machine-readable record naming which builder ran, from which source commit, through which steps, that a verifier checks against your policy. Reading that receipt is cheap next to a breach. It costs a few hundred milliseconds and one network call at the gate. One command already tells you whether a container came out of the workflow you expect and carries an unbroken SLSA provenance statement:

verify build provenance at the gate
$ gh attestation verify oci://ghcr.io/octo-org/octo-app:1.4.0 \
--repo octo-org/octo-app
Loaded digest sha256:9a2c...e1f7 for oci://ghcr.io/octo-org/octo-app:1.4.0
Loaded 1 attestation from GitHub API
✓ Verification succeeded!
sha256:9a2c...e1f7 was attested by:
REPO PREDICATE_TYPE WORKFLOW
octo-org/octo-app https://slsa.dev/provenance/v1 .github/workflows/release.yml@refs/tags/v1.4.0

Line the five handoffs up against their controls and the rest of the course has a shape: the Source track for the commit, hash-pinned lockfiles plus an SBOM and VEX for dependencies, an isolated hardened build emitting Build Level 3 provenance for the factory, keyless signing and digest pinning for publish, and policy verification at admission for deploy. Keep this attack tree open as you work through them. It tells you what each later lesson is buying you, and it explains why a control that only signs (SolarWinds had a valid signature) or only scans (SUNBURST had no CVE number) leaves a hole an attacker will find. Next comes the strongest control on that list: the SLSA levels themselves, and the provenance a verifier reads to prove a build was never touched.

Attack tree: one goal, five handoffs, one control each
Ship malicious code that inherits legitimate trust
attacker goal (STRIDE: Tampering → Elevation of privilege)
Source
Stolen creds, unreviewed or malicious commit (Codecov 2021)
Control: signed commits + 2-person review → SLSA Source track
Dependencies
Typosquat, maintainer hijack, poisoned update (event-stream, xz)
Control: hash-pinned lockfiles + SBOM + VEX
Build
Injection inside the trusted build (SolarWinds SUNSPOT)
Control: isolated, hardened build + provenance → SLSA Build L3
Publish
Mutable tag repointed, registry tamper (tj-actions 2025)
Control: keyless signing + pin by immutable digest
Deploy
Blind trust at admission = no evidence required
Control: verify signer identity + provenance at the gate
Trust flows only downstream: break the earliest link and every later one accepts the output. Each control makes one handoff produce verifiable evidence, and require it.
Quick check
01SUNSPOT sat on the SolarWinds build server, swapped a source file while the compiler ran, then put the clean file back afterwards. Which control most directly defeats this class of attack?
Incorrect — No. SUNSPOT worked inside the trusted build, so the backdoored DLL got SolarWinds' own real signature. Signing binds identity, not build integrity.
Correct — Isolation keeps one build from reaching into another, and provenance lets a verifier prove the built bytes came from the reviewed commit, which is what exposes the swap.
Incorrect — No. Lockfiles stop poisoned third-party updates like event-stream and xz. SUNSPOT altered first-party source during the build, which no dependency hash covers.
Incorrect — No. A custom implant like SUNBURST has no CVE number, so it walks through a vulnerability scan clean.
02The lesson pins every dependency to a cryptographic hash in a lockfile (pip install --require-hashes). What does that content-addressing actually change when a package is poisoned or quietly re-published?
Incorrect — A hash pins bytes and knows nothing about vulnerabilities. Scanning, plus SBOM and VEX, is what answers component risk.
Incorrect — A hash binds content, not a publisher. Identity is what a signature gives you, and the lesson keeps those two ideas apart.
Correct — The lesson shows pip stopping with 'THESE PACKAGES DO NOT MATCH THE HASHES', which converts a quiet compromise into a visible failure.
Incorrect — Content-addressing cannot prevent an upstream takeover. It catches the changed bytes downstream, at install time.
03Your workflow says uses: acme/deploy-action@v2, pinning the action to a floating tag. An attacker steals a maintainer token, force-pushes a malicious commit, and repoints tags v1 through v9 at it, exactly the tj-actions/changed-files pattern. What happens on your next pipeline run, and what pinning would have stopped it?
Incorrect — CI re-resolves the tag on every run and fetches whatever it points to now. No manual step is involved.
Correct — A tag is a movable pointer, so the repointed v2 fetches the payload. A commit SHA is calculated from the content and cannot be repointed.
Incorrect — Git follows the repointed tag without complaint. Nothing about a force-push makes the fetch fail.
Incorrect — The attacker repointed the whole range v1..v9, so v2 points at the malicious commit as well.

Try this

Work through “Stop trusting the label, start checking the evidence” yourself on a sandbox you can throw away, following the commands above in order. Then break one step deliberately and re-run, so you have seen the failure before it finds you.

Takeaway

The trap worth remembering here: signed does not mean safe. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related