Slimming images and layer hygiene

Find what makes an image big or leaky, and gate it in CI.

docker history of a 207 MB image shows a 20.5kB layer for rm -rf /var/lib/apt/lists/*, right above a 21.5MB layer for apt-get update. The cleanup step added bytes instead of removing them, and the package lists still ship to every host that pulls the image. Big images are slow to pull, slow to scan and carry more to patch; leaky images carry files nobody meant to publish. Both problems come from the same few habits, and both can be measured and gated in CI. This lesson does that with docker history, dive, Trivy and a small size budget.

A short recap of the mechanics, which "Layers and the build cache" in Docker for beginners teaches in full: each RUN, COPY and ADD adds a read-only layer, and deleting a file in a later layer only records a whiteout that hides it. The bytes stay in the layer that added them. Use the main lab VM; the lesson files go in ~/lab/slimming.

Find the big layers

Dockerfile.bloat installs curl the way many Dockerfiles still do, one command per instruction:

FROM debian:trixie-slim
RUN apt-get update
RUN apt-get install -y curl
RUN rm -rf /var/lib/apt/lists/*

Read history bottom up. The Debian base is 110MB. apt-get update downloaded 21.5MB of package lists into its own layer, apt-get install added 22.7MB, and the rm layer is 20.5kB: the whiteout entries plus a little metadata. The lists are hidden in the final filesystem and still present in the image. --no-cache forces apt to run here; without it a rebuild would reuse the cached layers.

Dockerfile.lean does the same job in one instruction and skips recommended packages, adding back the one recommendation that matters for curl, the CA certificates:

FROM debian:trixie-slim
RUN apt-get update \
 && apt-get install -y --no-install-recommends curl ca-certificates \
 && rm -rf /var/lib/apt/lists/*

One layer of 18.8MB instead of 44MB, because the lists are deleted before the layer is committed and --no-install-recommends leaves out packages curl does not need. DISK USAGE drops from 207MB to 165MB, and CONTENT SIZE, roughly what a pull transfers, from 53.5MB to 36.4MB. Your numbers will differ with the current Debian package versions. The same rule holds for every package manager: apk add --no-cache, pip install --no-cache-dir, npm ci followed by npm cache clean --force in the same RUN. A cache mount (RUN --mount=type=cache, covered in "BuildKit builds: stages, cache and mounts" in Docker in depth) goes further: the download cache never enters a layer and still speeds up the next build.

--squash is not the answer

Older guides fix the bloated file by flattening it with docker build --squash. On a default Docker 29 install the flag is still accepted, and ignored:

BuildKit prints a warning and builds the same four layers. Squashing belonged to the legacy builder with an experimental daemon. The warning names the replacement: a multi-stage build whose final stage copies only the result, so nothing deleted in an earlier stage is ever part of the image.

Score an image with dive

dive reads an image layer by layer and reports files that were added and then replaced or deleted later, the bytes that ship without being visible. In CI mode it applies thresholds and exits non-zero on failure. It normally talks to the Docker daemon through the socket, which would give the dive container root-equivalent access to the host ("The Docker socket and daemon hardening"); reading a docker save archive with --source docker-archive needs no socket. The image is pinned by digest and pulled through mirror.gcr.io:

The colour codes dive prints are stripped with sed for this page, and exit $rc hands dive's exit status, 1 here, to the CI job as the gate. Typed into your own shell, that exit closes the session, so paste this block (and the Trivy block below that ends with exit) inside ( ... ), where it ends only the subshell. dive found 25MB of waste: the package lists (written in one layer, hidden by the next), and debconf and dpkg files that the install rewrote. Efficiency 83.5% fails the 95% floor, and the 61% user-wasted ratio fails the default limit of 10%. Now the lean image:

Efficiency is 98.4%, and it still fails the default highestUserWastedPercent of 10%. The files listed are dpkg's status database and debconf's caches, which exist in the Debian base and are rewritten by any apt-get install; the old copies in the base layer count as waste. Every image that installs a package on a Debian base hits this. Set the threshold from what your images really look like, here 25%, and keep the efficiency floor, which is what catches lists and caches left in layers.

What the build context brings in

Size is the visible half. The other half is files that reach a layer by accident, and COPY . . is the usual way in. The app directory holds a five-line Dockerfile and a one-line app.py; give it the junk a real working copy has, a .env with a (fake) token and a .git directory:

FROM python:3.14-slim
WORKDIR /app
COPY . .
USER 10001
CMD ["python", "app.py"]

Everything in the directory is in /app, including the token file and 2MB of git objects. The warning on the first line comes from Buildx: it tries to record the current commit in the image provenance when the context has a .git directory, and this one is not a real repository. A secret scanner run on the saved image finds the token.:

Trivy's secret scanner matched the GitHub token format, masked the value in its report and named the instruction that added the file. --exit-code 1 turns any finding into a failed CI step. Trivy reads the archive from docker save, so it needs no access to the Docker socket either. Now add a .dockerignore (its basics are in "Writing a Dockerfile" in Docker for beginners):

.git
.env
*.pem
Dockerfile
.dockerignore

Only app.py reaches the image and the scan exits 0. A .dockerignore limits what the context sends; copying named files (COPY app.py ./) limits what a layer receives, and is the stronger of the two. A scanner is still worth running, because it checks the result, including layers the final filesystem hides:

FROM alpine:3.22
COPY app/.env /tmp/.env
RUN rm /tmp/.env

The file was deleted one step after it was copied, and Trivy still reports it, attributed to the COPY layer, because that layer ships. Credentials needed during a build belong in a secret mount, never in the context; "Build-time secrets and how images leak them" measures every place they can otherwise end up. If a scan like this finds a real credential in an image that was pushed, rotate it first.

A size budget

dive measures waste, not size. A budget catches the other regression: a new dependency, a base image that grew, a debug tool someone left in. check-size.sh compares the image's disk usage (the .Size field of docker image inspect, in bytes) with a limit:

#!/bin/sh
# check-size.sh IMAGE MAX_MB: fail when the image's unpacked size is over budget
img=$1 max=$2
bytes=$(docker image inspect --format '{{.Size}}' "$img") || exit 2
mb=$((bytes / 1000000))
if [ "$mb" -gt "$max" ]; then
  echo "FAIL $img: $mb MB, budget $max MB"
  exit 1
fi
echo "ok   $img: $mb MB, budget $max MB"

Set the budget a little above the current size and raise it on purpose when a change justifies it, in the same review. Compressed size is the better number when pull time is what you care about; it is the CONTENT SIZE column of docker image ls, or the sum of layer sizes in the pushed manifest.

Deterministic dependencies

Gates only mean something when two builds of the same commit produce the same contents. Install from lockfiles (npm ci with package-lock.json, pip install --require-hashes -r requirements.txt, go.sum), pin the base image by digest as described in "Tags, digests and promotion" in Docker in depth, and pin package versions where an unexpected upgrade would hurt. Debian's snapshot archive makes an apt-get install repeatable when that matters. BuildKit can also normalise file timestamps from SOURCE_DATE_EPOCH, which removes one more source of differences between rebuilds. In a pipeline, run the secret scan, the waste check and the budget against the built image, before the push, so a failing image never reaches a registry; "Docker in CI/CD: build to promote" in Docker in depth puts them in order with tests, signing and promotion.

Clean up

The dive and Trivy images stay in the image store; remove them with docker image rm when you no longer need them. In your own pipelines, pin the Trivy image by digest the way dive is pinned here: in March 2026 attackers published malicious Trivy images and force-pushed the trivy-action tags (CVE-2026-33634), and references by digest were not affected.

Try this

Work through “Clean up” yourself on a sandbox you can throw away, following the commands above in order. Then break one step deliberately and re-run, so you have seen the failure before it finds you.

Takeaway

If you keep one thing from slimming images and layer hygiene, keep “Clean up”. Decide now which check you will run when this shows up on a live system, and write it somewhere your team will find it.

Related