Images, build args, and layers

Secrets baked into layers you can still pull.

In short. Container images are tar layers with memory. A secret introduced in one layer remains recoverable from the image history even if a later layer deletes the file.

Container images are tar layers with memory. A secret introduced in one layer remains recoverable from the image history even if a later layer deletes the file. docker history, docker save, and public registries have all been used to recover keys that engineers thought they erased.

Build-args are especially treacherous: ARG TOKEN followed by RUN curl -H "Authorization: $TOKEN" may leave the value in layer metadata. Multistage builds help only if the secret never enters a layer you ship. BuildKit secret mounts (RUN --mount=type=secret) keep material out of the layer filesystem when used correctly.

See the problem

# anti-pattern demo (do not push)
printf 'FROM alpine\nARG TOKEN\nRUN echo $TOKEN > /tmp/t && rm /tmp/t\n' > /tmp/Df
DOCKER_BUILDKIT=0 docker build --build-arg TOKEN=supersecret -t leakdemo -f /tmp/Df /tmp
docker history leakdemo
docker save leakdemo | strings | grep -o supersecret | head -1

... /bin/sh -c echo supersecret > /tmp/t && rm /tmp/t
supersecret
# FAIL (expected) — rm does not erase history

Better patterns

Use BuildKit secret mounts for package tokens and private module credentials. Pull runtime secrets at start from a manager — do not bake them. Scan images with tools that look for high-entropy strings and known prefixes before push. Prefer distroless/minimal final stages so tooling that expected /tmp leftovers finds nothing.

# syntax=docker/dockerfile:1.6
FROM alpine
RUN --mount=type=secret,id=npm_token \
    NPM_TOKEN=$(cat /run/secrets/npm_token) \
    && npm config set //registry.npmjs.org/:_authToken=$NPM_TOKEN \
    && npm ci && rm -f ~/.npmrc

Going deeper

Multi-stage builds fail open when you COPY a builder stage wholesale. Copy only artifacts (COPY --from=build /out /app). Run scanners like Trivy or Grype in CI with secret content detectors enabled where available, and fail on high-entropy hits in layers.

Registry access tokens used in docker login on shared runners linger in root's config.json. Prefer short-lived registry cred helpers or cloud-native pull identities. Wipe runner workspaces between jobs.

Supply-chain note: a compromised base image can steal build secrets from BuildKit mounts if the RUN line is malicious. Pin base digests and review Dockerfiles like code that handles production keys — because it does.

Treat build secrets like production secrets in threat models. A pull request that adds a Dockerfile RUN line reading a mounted secret deserves the same review as a Vault policy change. Compromised dependencies in the build can exfiltrate mounts — pin versions and use locked package manifests.

For local developer builds, never reuse production tokens. Issue personal low-privilege tokens or use mock registries. The habit of "just once with the prod token" is how laptop malware becomes a registry-wide incident.

Try this

Build with BuildKit secret mount versus ARG on a throwaway Dockerfile; use docker history and strings to contrast.

echo supersecret > /tmp/tok
printf '# syntax=docker/dockerfile:1.6\nFROM alpine\nRUN --mount=type=secret,id=tok cat /run/secrets/tok > /dev/null\n' > /tmp/Df2
DOCKER_BUILDKIT=1 docker build --secret id=tok,src=/tmp/tok -t saferdemo -f /tmp/Df2 /tmp
docker history saferdemo
docker save saferdemo | strings | grep supersecret || echo 'no secret in layers'

...
no secret in layers
# SUCCESS — mount kept value out of image filesystem

Takeaway

Image layers remember. Never put secrets in ARG/ENV for shippable stages; use BuildKit secret mounts, clean contexts, and pre-push scanning.

Next: Kubernetes Secrets — what the platform object does and does not protect.

Related