CoursesDocker in depthImage layers & internals

Image layers & internals

How layers stack, cache, and get shared.

Intermediate14 min · lesson 1 of 30

An image isn't one file. It's a stack of read-only layers laid down in order, plus a small JSON (JavaScript Object Notation, a plain-text format for structured data) config that says how to run the thing. FROM picks the base image and brings all of its layers along. After that, every COPY, ADD or RUN that changes the filesystem adds exactly one layer on top, and so does a WORKDIR that has to create its directory. Each layer is named after the SHA-256 (Secure Hash Algorithm, 256-bit) hash of its own contents, a long fingerprint worked out from the bytes themselves. Instructions that only touch metadata (ENV, CMD, ENTRYPOINT, LABEL, USER, EXPOSE) still show up in the build, but they write a zero-byte layer that edits the config and nothing on disk.

That naming scheme is the whole trick. Say a coat check worked out your ticket number from the coat itself instead of handing you a random one. Two identical coats would come back with the same ticket, so the cloakroom could hang a single coat and point both tickets at it. Docker plays exactly that game with layers. If two images hold a byte-for-byte identical layer, it hashes to the same value, so it sits on disk once and comes down the wire once, however many images use it.

Reading an image back with docker history

You never have to guess what is inside an image. docker history replays the build for you, one row per step, showing the instruction that made each one and the bytes it added. Read it bottom to top: the base image sits at the bottom, your last instruction at the top.

terminal
$ docker image history payments-api:1.0
IMAGE CREATED BY SIZE
b2c3d4e5f6a1 CMD ["node" "server.js"] 0B
<missing> COPY . . 1.2MB
<missing> RUN npm ci --omit=dev 48MB
<missing> COPY package*.json ./ 4kB
<missing> WORKDIR /app 0B
<missing> FROM node:22-alpine 138MB

Those <missing> rows are not an error and not damage. BuildKit, the modern build engine, doesn't save a separate image for every step the way the old builder did, so only the finished image ends up with a real ID. Everything below it prints <missing>, both the base image's layers and the ones you built a minute ago. The command and the size still print, which is nearly always what you came for. Add --no-trunc when a row gets chopped off, and -H=false when you want raw bytes instead of friendly rounded sizes.

The config: what an image will do before you run it

Layers are the filesystem. The config is everything else, the runtime defaults baked in at build time: the entrypoint and command, environment variables, exposed ports, the working directory, the user the process runs as, and the ordered list of layer digests (each layer's content hash) that form the root filesystem. Reading it is how you learn what an image will do before you run it, which matters most for images you did not build yourself. The output below shows why you bother. You set the command, but the entrypoint came from the base image, and an empty user means this container runs as root. Both are the kind of thing you want to catch before you ship.

terminal
$ docker image inspect -f '{{json .Config}}' payments-api:1.0 | jq '{Entrypoint,Cmd,User,Env}'
{
"Entrypoint": ["docker-entrypoint.sh"],
"Cmd": ["node","server.js"],
"User": "",
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"NODE_VERSION=22.20.0"
]
}
$ docker image inspect -f '{{len .RootFS.Layers}} layers' payments-api:1.0
5 layers

A layer stores the difference, not the whole disk

Here is where people trip. A layer does not hold a complete filesystem. It holds the difference from the layer beneath it: the files added, the files changed, the files deleted, and nothing else. A Git commit works the same way, recording what moved rather than stashing a fresh copy of the whole repository. When you run a container, Docker stacks all those differences and shows them as one merged view through a union filesystem (on Linux that is usually the overlay2 storage driver). It works like sheets of transparent film on a light table: look down through the stack and you see one combined picture, with the top sheet winning wherever someone has drawn on it.

Running a container lays one more sheet on top: a thin writable layer. Reads fall through to whichever lower layer owns the file. The first write to any file copies it up into the writable layer, then edits the copy. That is copy-on-write, the same move as a shared document that forks you a private copy the moment you start typing. It is why containers start instantly (nothing is copied up front), and why every change a container makes stays trapped in that container and is thrown away when you remove it.

Union filesystem: read-only image layers plus one writable container layer
Image: read-only layers (shared, cached, pulled once)
FROM node:22-alpine
base layer, 138MB, content-addressed by SHA-256
COPY package*.json + RUN npm ci
dependency layer, a 48MB diff
COPY . .
app source layer, a 1.2MB diff
config (JSON)
entrypoint, cmd, env, user, ordered layer digests
Container: writable layer (copy-on-write, per container)
thin writable layer
added on docker run; first write copies the file up
discarded on docker rm
every change isolated here, then thrown away
terminal
$ docker run -d --name web alpine:3.20 sleep 1000
$ docker exec web sh -c 'echo hi > /tmp/note'
$ docker diff web
C /tmp
A /tmp/note
$ docker rm -f web
web

docker diff prints exactly what the writable layer changed against the image. A means added, C means changed, D means deleted. Here the container created /tmp/note, so /tmp comes back as changed and the file as added. Delete the container and the writable layer goes with it, note and all. Nothing you did leaked back into the image.

Ordering your Dockerfile is ordering the diffs

Because each layer is named after its own contents, a cached layer stays valid only while its inputs and every layer beneath it stay untouched. Change one, and Docker rebuilds that layer and everything above it. So the order of your instructions is really an order of diffs, and it decides how much of the build you get for free. Copy your dependency manifests and install them before you copy source code. An everyday code edit then busts only the top layer, and the expensive install stays cached.

Dockerfile
FROM node:22-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev
COPY . .
CMD ["node", "server.js"]
terminal
$ docker build -t payments-api:1.0 .
[+] Building 0.6s (10/10) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> [internal] load metadata for docker.io/library/node:22-alpine 0.3s
=> [internal] load build context 0.0s
=> => transferring context: 512B 0.0s
=> [1/5] FROM docker.io/library/node:22-alpine@sha256:9f3b... 0.0s
=> CACHED [2/5] WORKDIR /app 0.0s
=> CACHED [3/5] COPY package*.json ./ 0.0s
=> CACHED [4/5] RUN npm ci --omit=dev 0.0s
=> [5/5] COPY . . 0.2s
=> exporting to image 0.1s

Read that build output the way you read the history above. Steps 2 through 4 come back CACHED because their inputs did not move, so npm ci never runs at all. Only step 5, the source copy, rebuilds after your edit. A four-minute install collapses into a fraction of a second, entirely because of where two COPY lines sit.

The file you deleted is still in the image
Layers are immutable and stacked in order, so deleting a file in a later layer cannot shrink an earlier one. Docker writes a whiteout marker, a small tombstone telling the merged view to pretend the file is gone, while the real bytes stay in the earlier layer's diff, recoverable by anyone with docker save and tar. A 400MB archive you download in one RUN and rm in the next ships with the image forever. A secret you COPY in and delete later is wide open to everyone who pulls the image. Create and remove big or sensitive things inside a single RUN so they never become their own layer, or use multi-stage builds and BuildKit build secrets (RUN --mount=type=secret) so the secret is never committed to any layer at all.

Why shared layers matter on a busy host

Take a build farm that pulls the same alpine and debian bases a hundred times a day. If every image carried a private copy of those layers, disk would fill with duplicates and pull times would crawl. Content-addressed layers stop that cold. Two images that both start FROM alpine:3.20 reference the same layer digests, so the storage driver keeps one blob and points both images at it. That is why docker system df often shows an Image size far bigger than the Reclaimable figure: much of what looks enormous is shared. When you delete one image, Docker removes only the layers no other image still references. Knowing that sharing exists is the difference between pruning safely and yanking a layer another service still needs.

Cache invalidation runs on the same rules. Change an early Dockerfile line and every later layer rebuilds, even when those later RUN commands are identical, because the parent digest moved. That is the trade against one fat RUN that installs everything: a single big layer caches badly the moment any package version shifts, while many thin layers cache well right up until you touch an early instruction. Keep the stable work early, package installs before the COPY of app source, so a one-line code change rebuilds only the last layers. During a live incident, the engineer who can read docker history and say which layer grew after the deploy saves everyone hours of guessing.

Suppose the on-call engineer gets paged because a node is out of disk. The reflex is docker system prune -a. Hold off. List the images first, and which containers still use them. A dangling layer from a failed build, one no image name points at any more, is safe to drop. A tagged image still referenced by a stopped container you are keeping for forensics, with a volume mounted, is not. Shared layers make the arithmetic strange: freeing 200 MB of unique layers can look like a 2 GB image vanished from docker images while df barely moves. Measure with docker system df -v before and after, and reach for prune filters rather than a blind -a when the host also runs long-lived debug containers.

Try this

Run these on a lab engine (anything from Docker 24 onwards is fine). Read the sample output first so you know what a good result looks like before you lean on the command in production.

terminal
$ docker pull nginx:1.27-alpine
$ docker history --no-trunc nginx:1.27-alpine | head -n 8
IMAGE CREATED CREATED BY SIZE
sha256:9a1… 2 weeks ago /bin/sh -c #(nop) CMD ["nginx" "-g" "daem… 0B
sha256:9a1… 2 weeks ago /bin/sh -c #(nop) EXPOSE 80 0B
sha256:7c2… 2 weeks ago /bin/sh -c #(nop) COPY file:… in /etc/nginx 8.2kB
# each row is a layer; identical digests on two images are shared on disk
$ docker system df
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 12 4 3.1GB 1.4GB (45%)
Containers 4 4 212MB 0B (0%)
Local Volumes 3 2 480MB 120MB (25%)

Takeaway

Layer digests are the ground truth here. Tag names can be repointed, the totals in docker images count shared blobs twice, and a file you deleted still weighs whatever it always weighed. When a number looks wrong, go back to the hashes: docker history for which step grew, docker system df -v for what is genuinely reclaimable, and docker diff for what a running container has changed since it started.

Quick check
01You COPY a 500MB dataset into an image, use it during the build, then rm it in a later RUN instruction. How big does the finished image end up?
Incorrect — The rm runs in a later layer, so the most it can do is drop a whiteout marker over the file. All 500MB is still sitting in the earlier layer's diff.
Correct — Layers are frozen diffs stacked in order, so a delete higher up hides the file from the merged view but cannot shrink a layer below it. Download and delete inside one RUN if you want the space back.
Incorrect — Nothing collects garbage across layers during a build. Each layer is sealed the moment it is written.
Incorrect — overlay2 or another driver changes how layers merge at runtime, not whether a deleted file's bytes stay committed in the layer below.
02Two images on the same host both contain a byte-for-byte identical base layer. How many copies land on disk, and how many times does it come over the network?
Incorrect — Identical content is shared between images, never duplicated per image.
Incorrect — Tags name whole images. A layer's identity comes from the hash of its bytes, and no tag changes that.
Correct — Content addressing collapses identical layers into one blob on disk and one download.
Incorrect — There is no second copy to compress. The shared layer is stored exactly once.
03Using the Dockerfile above, which copies package*.json and runs npm ci before COPY . ., you edit one line of server.js and rebuild. What does Docker do?
Correct — A layer keeps its cache until its own inputs or something beneath it changes, so only the source copy and anything above it get rebuilt.
Incorrect — Layers below the change keep their cache. Only the changed layer and the ones above it rebuild.
Incorrect — npm ci sits below the source copy and its inputs (package*.json) did not move, so it stays cached.
Incorrect — server.js is part of the COPY . . input, so that layer's contents changed and it has to be rebuilt.

Related