CoursesDocker for beginnersThe layered filesystem & build cache

The layered filesystem & build cache

Why instruction order makes builds fast.

Beginner12 min · lesson 10 of 16
In plain terms
Picture sheets of tracing paper stacked on top of each other, each adding a few marks. Docker builds an image the same way, one sheet per instruction, and it reuses the bottom sheets it already has — so if only the top sheet changed, it redraws just that one.

A Docker image is a sealed tray of frozen food. Everything the meal needs is already inside it, portioned and finished, and nothing about it changes once the factory wraps it. That is what read-only means: fixed. Your app, the libraries it borrows and the slice of operating system it sits on are all in there, packed once. A container is that tray heated up and served. One image, many meals, every one of them the same. Docker does not produce the tray in a single motion, though. It builds up in thin passes, each one laid over the last, and each pass is called a layer. Stack a dozen clear plastic sheets, put a few marks on each, then look down through the pile: the marks combine into one picture. An image works exactly like that.

You write those passes down in a Dockerfile, which is a recipe card for your app. A list of steps, in order, that turns out the same dish every time somebody follows it. It is a plain text file with one build instruction per line, and every instruction that changes the files inside the image adds a layer. FROM picks your starting point: a base image someone else already published, so you begin with a working system instead of an empty box. COPY carries your files in. RUN runs a command during the build and bakes whatever it produced into a new layer. Docker names each layer after its exact contents, so two images that both start FROM node:22-alpine share that base on disk rather than keeping two copies of it. And when you see the word pull, read it as download from a registry, the online shop where images live. Docker Hub is the big one.

Every layer inside an image is read-only. Frozen solid. The moment you start a container, Docker slips one extra layer on top, and that thin top layer is the only surface the running program can write to. Log files, uploads, temporary scratch: all of it lands there. Remove the container and that layer goes in the bin with it. That is the whole reason anything a container writes to disk disappears the moment the container does.

An image is a stack of layers
Read it bottom to top. Each instruction adds a layer, the base is shared between images, and only the writable layer belongs to a single container.

See the layers yourself

Here is a small app you can build on your own machine. Four files in one folder: a package.json listing a single dependency (a package of ready-made code your app borrows), the package-lock.json that npm writes beside it (it pins the exact version of every package so two builds come out identical), a server.js holding your code, and the Dockerfile below. The name after -t, payments-api:1.0 here, is a tag: a plain label plus a version, so you have something to point at later.

Dockerfile
FROM node:22-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev
COPY . .
EXPOSE 3000
CMD ["node", "server.js"]

Three of those instructions are new. WORKDIR sets the folder that every later command runs inside, so you never have to spell out the full path. EXPOSE writes down which port the app listens on. CMD names the command Docker runs when a container starts. Build it now and read what scrolls past. Every line that opens with a step number is one layer being made, in the order you wrote them.

terminal
$ docker build -t payments-api:1.0 .
[+] Building 9.3s (11/11) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 132B 0.0s
=> [internal] load metadata for docker.io/library/node:22-alpine 0.8s
=> [internal] load .dockerignore 0.0s
=> [1/5] FROM docker.io/library/node:22-alpine@sha256:9fcc1a... 1.7s
=> [internal] load build context 0.0s
=> => transferring context: 1.44kB 0.0s
=> [2/5] WORKDIR /app 0.1s
=> [3/5] COPY package*.json ./ 0.0s
=> [4/5] RUN npm ci --omit=dev 5.6s
=> [5/5] COPY . . 0.0s
=> exporting to image 0.2s
=> => writing image sha256:4b1e2f... 0.0s
=> => naming to docker.io/library/payments-api:1.0 0.0s

Look at the lines marked [n/5]. Each one is a layer, built in order, starting from the base image. Step [4/5], the npm ci install, ate most of the clock. npm ci means a clean install of your dependencies read straight from that lock file, nothing improvised. That is precisely the step you want Docker to skip on every build after the first. You can also list the finished layers with their sizes:

terminal
$ docker history payments-api:1.0
IMAGE CREATED CREATED BY SIZE
4b1e2f9c8a3d 2 minutes ago CMD ["node" "server.js"] 0B
<missing> 2 minutes ago EXPOSE map[3000/tcp:{}] 0B
<missing> 2 minutes ago COPY . . # buildkit 12.3kB
<missing> 2 minutes ago RUN /bin/sh -c npm ci --omit=dev # buildkit 4.9MB
<missing> 2 minutes ago COPY package*.json ./ # buildkit 1.4kB
<missing> 2 minutes ago WORKDIR /app 0B
<missing> 3 weeks ago /bin/sh -c #(nop) CMD ["node"] 0B
<missing> 3 weeks ago /bin/sh -c #(nop) ADD file:... in / 8.4MB

Your WORKDIR, COPY and RUN steps sit near the top of that list, with the base image's own layers underneath them. The <missing> in the ID column is nothing to worry about. It only means those lower layers carry no tag of their own. Now read the sizes. EXPOSE and CMD weigh 0B because they record a setting and touch no files. npm ci is where the real weight went.

Change one line, rebuild, watch the cache

Docker hangs on to every layer it has ever built. On the next build it walks down your Dockerfile in order and reuses each stored layer for as long as that instruction's inputs are unchanged. The first instruction whose inputs did change gets rebuilt, and so does every layer sitting above it, because each layer is stacked on the one below. So edit a single line of server.js and build again:

terminal
$ docker build -t payments-api:1.1 .
[+] Building 0.6s (11/11) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> [internal] load metadata for docker.io/library/node:22-alpine 0.4s
=> [internal] load .dockerignore 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 1.44kB 0.0s
=> CACHED [2/5] WORKDIR /app 0.0s
=> CACHED [3/5] COPY package*.json ./ 0.0s
=> CACHED [4/5] RUN npm ci --omit=dev 0.0s
=> [5/5] COPY . . 0.0s
=> exporting to image 0.1s
=> => naming to docker.io/library/payments-api:1.1 0.0s

Steps 2, 3 and 4 come back marked CACHED. Docker reused them untouched. Only [5/5] COPY . . actually ran, because that is the one step that noticed your edited file. The whole rebuild finished in well under a second and npm ci never woke up. That is your payoff for copying package*.json and installing before you copy the rest of the source. The install sits below your code, so a code edit can never reach down and disturb it.

Now turn the order upside down and watch it fall over. This version copies everything first, then installs:

Dockerfile
FROM node:22-alpine
WORKDIR /app
COPY . .
RUN npm ci --omit=dev
EXPOSE 3000
CMD ["node", "server.js"]

Change one line of source and rebuild with this one. COPY . . now sits below the install. Touch any file at all and that lower layer changes, which drags npm ci and everything stacked above it into the rebuild:

terminal
$ docker build -t payments-api:bad .
[+] Building 5.9s (10/10) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> [internal] load metadata for docker.io/library/node:22-alpine 0.4s
=> [internal] load .dockerignore 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 1.44kB 0.0s
=> CACHED [2/4] WORKDIR /app 0.0s
=> [3/4] COPY . . 0.0s
=> [4/4] RUN npm ci --omit=dev 5.4s
=> exporting to image 0.3s
=> => naming to docker.io/library/payments-api:bad 0.0s

Step [4/4] npm ci ran again. That is 5.4 seconds burned on a one-line edit to a file with nothing to do with your dependencies. Scale it up to a real project carrying a few hundred packages and you are choosing between a one-second rebuild and a two-minute one, every single time you fix a typo. Same app, same commands, same finished image. Two instructions swapped places.

Rebuilt the image, now run it

Rebuilding an image leaves anything already running completely alone. The old container keeps serving, and it keeps its grip on whatever ports it claimed at startup. Try to start the new image on the same port and Docker refuses:

terminal
$ docker run -d -p 3000:3000 payments-api:1.0
c0ffeecafe12ab34cd56ef78901234567890abcdef1234567890abcdef123456
$ docker run -d -p 3000:3000 payments-api:1.1
docker: Error response from daemon: driver failed programming external
connectivity on endpoint zealous_khorana (7f3c...): Bind for
0.0.0.0:3000 failed: port is already allocated.

That message comes from the Docker daemon, the background program that does the actual work of building images and running containers. The docker commands you type are orders handed over to it. Read the last line: port is already allocated. A port is a numbered doorway on your machine for network traffic, and one program at a time gets a given doorway. The -p 3000:3000 flag wires port 3000 on your machine through to the container. The container running :1.0 still owns door 3000, so the new one has nowhere to land. Find it, remove it, then start the replacement:

terminal
$ docker ps --format '{{.ID}} {{.Image}} {{.Ports}}'
c0ffeecafe12 payments-api:1.0 0.0.0.0:3000->3000/tcp
$ docker rm -f c0ffeecafe12
c0ffeecafe12
$ docker run -d -p 3000:3000 payments-api:1.1
9a8b7c6d5e4f01234567890abcdef1234567890abcdef1234567890abcdef1234
A deleted file still ships inside the image
Every layer is kept, so a file you add in one instruction and delete a few lines later is still sitting there in the earlier layer, and anyone holding the image can dig it back out. Copy in a password or an access key (a string of characters that proves who you are to another service), delete it further down, and you have hidden nothing at all. Keep secrets out of a Dockerfile in the first place. A secret in any layer is a secret in the whole image.

Three working habits fall out of everything above, and they are worth a sticky note on your monitor. The first is about order, the second about trust, the third about size.

Order. Copy the dependency files, install them, then copy the rest of your source. Change your app code and Docker hands the cached install straight back to you. Change package.json and it correctly throws that layer away and reinstalls, which is what you want, because your dependencies genuinely did change. Put COPY . . near the top and every typo you fix rebuilds the universe.

Trust. A cached layer is fast, not fresh. If a RUN step downloads whatever counts as "latest" out on the internet, the cache will happily serve you a copy from three weeks ago and never mention it. Pin versions so the answer cannot drift under you. When you truly need Docker to forget everything and start clean, docker build --no-cache is the honest hammer for it.

Size. Deleting a file in a later layer does not claw the bytes back out of the earlier one, so they ride along in every copy of the image anybody downloads. Multi-stage builds, which come up in a later course, fix that class of problem properly. For now, keep junk out of your early layers and keep .dockerignore honest, so node_modules and .git never enter the build context at all.

Try this

Build once, make a tiny edit to a source file, build again, and read which steps come back CACHED. Thirty seconds of watching that teaches the layer cache better than any explanation.

terminal
docker build -t layerdemo:1 .
# edit a source file, then:
docker build -t layerdemo:1 .
docker history layerdemo:1 | head -n 8
output
[+] Building ...
=> CACHED [2/5] WORKDIR /app
=> CACHED [3/5] COPY package*.json ./
=> CACHED [4/5] RUN npm ci
=> [5/5] COPY . .
IMAGE CREATED BY SIZE
layerdemo:1 CMD ["node" "server.js"] 0B
<missing> COPY . . 12kB
<missing> RUN npm ci 80MB

Takeaway

Each instruction is a layer, and Docker rebuilds from the first changed layer all the way up. Install your dependencies before you copy your app code, and a one-line edit keeps costing you a second instead of a coffee break.

Quick check
01You change one line in server.js and rebuild. Docker prints CACHED next to WORKDIR, COPY package*.json and RUN npm ci, then rebuilds COPY . .. Why did npm ci get skipped?
Correct — Docker reuses cached layers until it meets the first changed instruction, then rebuilds that layer and everything on top of it. The npm ci layer sits below COPY . ., so a source edit never reaches down that far.
Incorrect — No. A RUN step rebuilds the moment any instruction before it changes. This one survived only because nothing below it moved.
Incorrect — No. RUN happens at build time and is baked into a read-only image layer. The writable layer only shows up once a container is running.
Incorrect — No. The base image has never heard of your packages. The saving comes entirely from the order of your own COPY and RUN lines.
02A running container writes a new file to disk. Where does that file actually go, and what happens to it when you delete the container?
Incorrect — Image layers are read-only and settled at build time. A running container cannot add one.
Incorrect — The base is read-only and shared. A container's writes never reach it.
Correct — That top layer belongs to a single container, so its contents go when the container goes.
Incorrect — Writes at runtime never travel back into your Dockerfile or into any future build.
03A teammate's Dockerfile runs COPY secrets.txt ., uses the file in a RUN step, then runs RUN rm secrets.txt a few lines further down. They tell you the secret is safe because the finished image no longer has it. Are they right?
Incorrect — The rm only stacks one more layer on top. The layer that brought the file in still holds it.
Correct — Every layer is kept, so a secret in any layer is a secret in the whole image.
Incorrect — Same build or not, the layer that added the file is retained and can be extracted.
Incorrect — The file is recoverable from the image itself, private registry or not.

Related