CoursesSoftware supply chain in depthHermetic & reproducible builds

Hermetic & reproducible builds

Pinned inputs, isolation, bit-identical output.

Advanced30 min · lesson 3 of 15

In March 2024 a Microsoft engineer went looking for half a second of unexplained delay when logging into a server over SSH (Secure Shell, the standard way to open a remote command line). That half second led him to a backdoor in XZ Utils, a compression library that ships on nearly every Linux machine. It was catalogued as CVE-2024-3094 (CVE, Common Vulnerabilities and Exposures, is the public register of known flaws). The malicious code had gone out weeks earlier in releases 5.6.0 and 5.6.1, and here is the part worth sitting with: it existed only inside the release tarballs, the packaged .tar.gz files people download, and never appeared anywhere in the project's Git history. The attacker got in through the build. A doctored helper file called build-to-host.m4 and a scrambled payload rode along in the tarball that autotools produces (autotools being the script generator that turns a project's source tree into a ready-to-compile package), while the source everyone reviewed stayed clean. The trick worked because almost nobody rebuilds a release from source and checks that the bytes they get back match the bytes that were published. Two build properties would have exposed it. A reproducible build lets an outsider rebuild the release and compare it byte for byte. A hermetic build keeps anything outside the declared, reviewed inputs from reaching the build at all. Provenance, which fills the rest of this section, is only ever as trustworthy as the build it describes.

Reproducible: same source in, the same bytes out

A recipe that says 350 degrees for 40 minutes should give you the same cake in any kitchen. A build is reproducible when it holds to that same standard: the same source and the same declared inputs produce output that is identical byte for byte, whoever runs it, on whatever machine, at whatever hour. It is a strong claim, and most toolchains fail it out of the box. Watch two archives of a directory nobody touched come out different.

terminal — nondeterministic by default
$ tar czf app.tar.gz build/ ; sha256sum app.tar.gz
9f2c0b6b1d8e4a77c0b2e1f5a3d9c8b41e7f0a2c6d5b3e9f1a4c7d0b2e5f8a1c3 app.tar.gz
$ sleep 2
$ tar czf app.tar.gz build/ ; sha256sum app.tar.gz
41ab77e0c3d9f2b5108c6a4e7d1b0f93a2c5e8d7b6f4a1c0e3d9b2f5a8c1e4d70 app.tar.gz
# ^ same files in build/, different bytes on disk

Nothing in build/ changed, yet the two digests disagree. A digest is the sha256 fingerprint printed above, a short string that changes completely if a single byte of the file changes. The nondeterminism here, meaning the build refusing to settle on one answer, is baked into everyday tools. gzip writes the source filename and the current time into its header. tar records each file's mtime (modification time), its uid and gid (the numeric user and group that own it), and whatever order the filesystem happened to return when the directory was listed. C compilers embed __DATE__, the absolute path of the directory you built in, and symbol orderings that shift with your locale settings. Every one of those is a clock or a machine detail leaking into the file you ship.

Pinning the clock: SOURCE_DATE_EPOCH and a fixed toolchain

The Reproducible Builds project settled on one lever for the timestamp problem: SOURCE_DATE_EPOCH, an environment variable holding a UNIX timestamp (a plain count of seconds since the start of 1970, which is how computers store a moment in time). Tools that honor it clamp any timestamp newer than that value down to it, so "now" stops leaking in. Set it from the source itself, the date of the commit, rather than from the wall clock. Then pin whatever nondeterminism your archiver has left.

pack.sh — pinned and deterministic
export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) # commit time, not wall clock
tar --sort=name --format=posix \
--pax-option='exthdr.name=%d/PaxHeaders/%f,delete=atime,delete=ctime' \
--mtime="@$SOURCE_DATE_EPOCH" \
--owner=0 --group=0 --numeric-owner \
-cf app.tar build/
gzip -n -9 < app.tar > app.tar.gz # -n: omit filename + timestamp
sha256sum app.tar.gz
# --- run the whole script twice, an hour apart, on two machines ---
$ bash pack.sh ; bash pack.sh
b7e0f3a2c14d59f6e8a1b0c3d7f2a5e9c4b6d1f8a0c2e5b3d7f9a1c4e6b8d0f32 app.tar.gz
b7e0f3a2c14d59f6e8a1b0c3d7f2a5e9c4b6d1f8a0c2e5b3d7f9a1c4e6b8d0f32 app.tar.gz
# ^ bit-for-bit identical

Run that script twice, an hour apart, on two different machines, and the two sha256sum lines come out identical. Each flag closes one leak. --sort=name fixes the file order. --mtime with SOURCE_DATE_EPOCH fixes the time. --owner=0 --group=0 --numeric-owner fixes ownership. gzip -n stops gzip stamping a name and a timestamp into its header. Pinning the toolchain matters every bit as much, because the digest only holds if everyone compiles with the same compiler version, and code generation and symbol ordering drift between releases. This is where reproducibility sits alongside the rest of the section. A signature tells you who produced an artifact. Provenance tells you how it was built. A reproducible build lets anyone confirm that "how" by rebuilding and comparing, taking nobody's word for it. Debian and Arch run rebuilder farms that do exactly this at scale, flagging any published binary that fails to match a fresh rebuild.

Reading a mismatch with diffoscope

A digest that does not match tells you about as much as a smoke alarm going off in an empty house: something is wrong somewhere, good luck finding it. diffoscope walks you to the smoke. It unpacks both artifacts recursively (archives, filesystem images, ELF sections, which are the labelled chunks inside a compiled Linux binary, even gzip streams buried inside other files) and prints a readable diff of the first thing that genuinely differs.

terminal — diffoscope on the mismatch
$ diffoscope app-1.tar.gz app-2.tar.gz
--- app-1.tar.gz
+++ app-2.tar.gz
├── filetype from file(1)
│ │ @@ -1 +1 @@
│ │ -gzip compressed data, was "app.tar", last modified: Thu Jul 16 09:41:02 2026
│ │ +gzip compressed data, was "app.tar", last modified: Thu Jul 16 09:41:04 2026
├── app.tar
│ ├── file list
│ │ @@ -1,3 +1,3 @@
│ │ --rw-r--r-- 0 alice alice 512 2026-07-16 09:41:02 build/main.o
│ │ +-rw-r--r-- 0 alice alice 512 2026-07-16 09:41:04 build/main.o

Here the diff points straight at timestamps. The gzip header's "last modified" and each file's tar mtime sit two seconds apart. That is a determinism bug rather than tampering, and SOURCE_DATE_EPOCH fixes it. diffoscope earns its keep on the uglier cases. If it turned up an extra object file, a changed symbol table, or an embedded build path like /home/alice/…, you would be looking at either an input that got in undeclared or a genuine injection, and you would want to know which one before you signed anything.

Hermetic: sealed off, every input pinned

Hermetic is the old word for airtight, the way a sealed preserving jar keeps the outside out. Reproducibility is about consistency; hermeticity is about control, and having one does not hand you the other. A build is hermetic when it declares every input up front and runs sealed off from everything else: no network, no ambient host tools, no undeclared files lying around, no reading of the clock. Nothing outside the reviewed input set can influence or contaminate what comes out. That is what makes provenance believable at all, because a builder can only honestly attest to inputs it fully controls. SLSA (Supply-chain Levels for Software Artifacts) already requires an isolated builder emitting unforgeable provenance at Build L3. Its v1.0 spec lists hermetic and reproducible builds above that baseline, as future directions that harden provenance further by shrinking what an undeclared input could ever smuggle in. Two systems enforce hermeticity seriously today. Nix builds each derivation (its term for one build step plus its declared inputs) in a sandbox with no network access, makes a fixed-output derivation declare its content hash up front, and names every output after a hash of all its inputs.

terminal — Nix hermetic build + rebuild check
$ nix build .#app --rebuild
$ nix path-info ./result
/nix/store/2q8v6h7k1r0mxz3d9c4b5a1f8s6g7h2j-app-1.0
# --rebuild builds a second time in a fresh, network-less sandbox and
# aborts if the result is not bit-identical. On a nondeterministic output:
$ nix build .#app --rebuild
error: derivation '/nix/store/…-app-1.0.drv' may not be deterministic:
output '/nix/store/2q8v…-app-1.0' differs from
'/nix/store/2q8v…-app-1.0.check'

nix build --rebuild is a reproducibility check wired into the tool itself. Bazel takes the same line for its actions: a sandbox that denies network by default, and external dependencies pinned by sha256. The trade-off is real. Hermetic builds run slower and feel stricter, because every dependency has to be vendored or hash-pinned instead of pulled over the network whenever it is convenient, and reproducibility becomes ongoing whack-a-mole against new sources of nondeterminism. That cost is precisely what buys you a build a compromised worker cannot quietly rewrite.

Bazel — .bazelrc + MODULE.bazel (inputs pinned by hash)
# .bazelrc — force hermeticity on every build
build --incompatible_strict_action_env # fixed PATH, scrub the ambient env
build --sandbox_default_allow_network=false # actions get no network
# MODULE.bazel — every external input pinned by content hash
bazel_dep(name = "abseil-cpp", version = "20240722.0")
http_archive = use_repo_rule("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "zlib",
sha256 = "9a93b2b7dfdac77ceba5a558a580e74667dd6fede4585b91eefb60f03b72df23",
strip_prefix = "zlib-1.3.1",
urls = ["https://zlib.net/fossils/zlib-1.3.1.tar.gz"],
)
# A clean rebuild yields the same binary digest:
$ bazel build //:app && sha256sum bazel-bin/app
$ bazel clean && bazel build //:app && sha256sum bazel-bin/app
3c1d9a02f7e5b4c8d0a6f1e3b9c7d2a5… bazel-bin/app
3c1d9a02f7e5b4c8d0a6f1e3b9c7d2a5… bazel-bin/app

An http_archive with no sha256, or a curl | bash sitting inside a build step, is the exact hole both tools were built to close: an input nobody hashed, free to change under you between one build and the next. That is the class of gap the XZ tarball walked through. When a rebuild's digest fails to match, diffoscope plus a short decision tree tells you whether you are looking at a dull determinism bug or something worth escalating.

Rebuild digest doesn't match: what is it?
digest ≠ signed release digest
run diffoscope on the two artifacts
only mtimes / gzip header
determinism bug
pin SOURCE_DATE_EPOCH; tar --sort --mtime; gzip -n
absolute build paths (/home/…)
path leak
gcc -ffile-prefix-map=; build in a fixed path
locale-dependent ordering
environment leak
LC_ALL=C; sort inputs; pin the toolchain version
extra files / changed symbols
integrity concern
unpinned input changed; treat as possible injection
Most mismatches are dull determinism bugs. The last branch is the one that matters: a difference you cannot explain from your own build settings means an input you did not control.
Container images stamp times into places the digest can see
A perfectly reproducible artifact wrapped inside a non-reproducible image still leaves you with an image digest that moves. OCI (Open Container Initiative, the standard format for container images) builds write a creation time into the image config and a modification time into every layer, so two docker builds of identical inputs hand you different manifest digests. BuildKit honors SOURCE_DATE_EPOCH from v0.11 onward, but normalizing the per-file layer timestamps as well needs the exporter option rewrite-timestamp=true (BuildKit v0.13+), for example --output type=image,name=…,rewrite-timestamp=true. Leave it off and the layer timestamps keep wandering. Pin the base image by digest too, FROM alpine@sha256:…, never FROM alpine:latest, or your hermeticity is standing on a floor that moves.
Quick check
01Two independent rebuilds of the same tagged source come out byte-identical, and the digest matches the signed release. What have you actually proved?
Incorrect — No. Reproducibility says nothing about component risk, which is what SBOMs (Software Bills of Materials) and scanning answer. A backdoored build reproduces its backdoor perfectly.
Correct — That trust-nobody, independently checkable link between source and binary is what reproducibility buys you, sitting next to signing (who) and provenance (how).
Incorrect — No. If a pinned input is poisoned, as the XZ tarball was, a reproducible build reproduces the backdoor identically. Reproducibility proves consistency, not innocence.
Incorrect — No. That is hermeticity, a separate property. A build can be perfectly reproducible while still pulling inputs over the network, as long as everyone pulls the same bytes.
02What does SOURCE_DATE_EPOCH do in a reproducible build, and where should its value come from?
Incorrect — It does not strip timestamps. It clamps any timestamp newer than its value down to that fixed value.
Correct — The lesson sets it with git log -1 --pretty=%ct so "now" stops leaking into the artifact.
Incorrect — It deals with timestamps. Symbol-ordering nondeterminism is handled by pinning the toolchain and fixing the locale.
Incorrect — It has nothing to do with signing or Rekor. It is a build-determinism lever for timestamps.
03A rebuild's digest doesn't match the signed release. You run diffoscope, and the only difference it surfaces is an absolute path like /home/alice/build/main.o baked into the binary. Per the lesson's decision tree, what is this and how do you fix it?
Incorrect — That is the mtime and gzip-header branch. An embedded build path is a different failure from a clock leak.
Incorrect — That branch covers differences you cannot explain from your own build settings, like extra files or changed symbols. A build path is fully explainable.
Correct — The decision tree sends absolute build paths to the path-leak branch, fixed with -ffile-prefix-map= and a fixed build location.
Incorrect — A path leak is fixable where it stands. It does not force a wholesale move to a different build system.

The next lesson takes the digest you now know how to reproduce and turns these build properties into evidence you can hand to someone else: the provenance a SLSA builder emits, and how to read the predicate that records which pinned inputs and which toolchain version produced that exact digest.

Try this

Run tar czf app.tar.gz build/ ; sha256sum app.tar.gz on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: container images stamp times into places the digest can see. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related