Docker interview questions
Prep Docker interviews from image layers and ENTRYPOINT through overlayfs, capabilities, and real escape paths — tagged Beginner to Expert so you can match the room.
Levels run Beginner → Intermediate → Advanced → Expert. Answers are phrased the way you would say them in an interview; Advanced and Expert answers add the deeper reasoning, a diagram where it helps, and the follow-up an interviewer often asks next.
Core concepts
An image is the immutable, layered filesystem template. A container is a running — or stopped — instance of that image with its own process tree, network namespace, and a thin writable layer on top.
docker images docker ps -a docker run -d --name web nginx:1.30Link to this question
Each Dockerfile instruction that changes the filesystem produces a content-addressed layer. Layers get cached and shared across images, so rebuilds skip layers whose instruction and inputs haven't changed.
docker history nginx:1.30
docker image inspect nginx:1.30 --format '{{json .RootFS.Layers}}'Link to this questionBoth put files into the image. ADD also auto-extracts local archives and can fetch remote URLs. I'd prefer COPY for local files and use ADD for tar extraction or a remote HTTPS or Git source pinned with --checksum.
COPY package.json package-lock.json ./ COPY src/ ./src/ # ADD for local .tar auto-extract, or a remote URL/Git source with --checksumLink to this question
The CLI talks to the daemon — or the containerd path — which pulls the image if needed, creates a writable layer, sets up namespaces and cgroups, then starts your process as PID 1 in that sandbox. There's no hypervisor boot.
There's no VM boot here. dockerd — or a containerd client — allocates a writable overlay upper dir, clones namespaces, applies cgroup limits, then execs the entrypoint as PID 1. If PID 1 is a shell wrapper, SIGTERM may never reach the app — I'd prefer exec-form ENTRYPOINT or --init. Failures at pull, storage driver, or name conflicts show up before the process starts; runtime crashes after that show up in docker logs and inspect State.
No VM boot — start-up is milliseconds because isolation is kernel namespaces, not hardware virtualization.
docker pull nginx:1.30
docker create --name web nginx:1.30
docker start web
docker inspect web --format '{{.State.Pid}} {{.HostConfig.NetworkMode}}'Interviewer often follows with: Why might SIGTERM from docker stop not reach your app?
Link to this questionLower dirs are the image layers — read-only. An upper dir holds container writes. A merged view presents both; deletes show up as whiteouts in the upper layer. That's why image layers stay shared and containers stay thin.
Lowerdirs are immutable image layers; the upperdir captures creates, modifies, and whiteouts for deletes. The merged mount is what the container sees. Writing into the container without a volume grows the upperdir on the Docker host disk — I've watched that fill a box overnight. Read-only rootfs forces writes onto explicit mounts. Storage driver mismatches and backing filesystem limits like xfs ftype still bite during installs. On Engine 29+ fresh installs, the containerd image store uses the overlayfs snapshotter instead of the overlay2 driver, with the same lower/upper/merged model.
docker info | grep -i 'storage driver\|backing filesystem'
docker inspect web --format '{{.GraphDriver.Name}} {{json .GraphDriver.Data}}'Interviewer often follows with: How do whiteouts interact with a multi-stage COPY --from?
Link to this questioncontainerd is the industry CRI/runtime daemon that pulls images, manages snapshots, and starts containers via runc. Docker Engine is the UX and API layer — CLI, build, Compose networking — that typically drives containerd underneath.
containerd owns image pull, snapshotters, and task lifecycle via runc or crun. Docker Engine adds the familiar API, BuildKit, Compose-friendly networking, and UX. Since dockershim was removed in v1.24, Kubernetes speaks CRI to containerd or CRI-O, and reaches Docker Engine only through the external cri-dockerd adapter. Fresh Engine 29 installs also keep images in the containerd image store. On nodes with both, confusing docker ps with the CRI's containers is a classic ops mistake — different stores and namespaces. Debugging production clusters for me means ctr/crictl more than docker.
docker version ctr version # containerd CLI, if installed # Kubernetes talks to containerd (or CRI-O) — not dockerd — via the CRI
Interviewer often follows with: Which CLI do you use to see pause containers on a kubelet node?
Link to this questionIt replaces Docker's classic storage drivers like overlay2 with containerd's image store and snapshotters. Since Engine 29 it's the default on fresh installs; upgraded hosts keep overlay2 until you switch. It lets you build and hold multi-platform images and images with attestations locally.
Switching is a daemon.json feature flag plus a daemon restart. The catch: images and containers created under the other backend become hidden. They stay on disk, but you push them to a registry or docker save them first if you need them. containerd keeps its data in a path separate from the Docker data root, so a custom data-root doesn't cover it and disk triage that only looks under /var/lib/docker misses it. docker info reports driver-type io.containerd.snapshotter.v1 when it is active. The fresh-install default doesn't apply to daemons configured with userns-remap.
docker info -f '{{ .DriverStatus }}'
# [[driver-type io.containerd.snapshotter.v1]]
# /etc/docker/daemon.json
{ "features": { "containerd-snapshotter": true } }Interviewer often follows with: You switch an existing host to the containerd store and every image disappears. What happened, and how do you recover?
Link to this questionI'd order the Dockerfile so stable steps come first. Copy dependency manifests and install packages before copying source. One changed instruction invalidates that layer and everything after it.
Build cache is instruction-keyed top-down. COPY . . early forces dependency installs to rebuild on every commit — I've wasted hours of CI on that. The pattern I'd use: copy lockfiles → install deps → copy source → build. BuildKit helps with remote cache and --mount=type=cache for package managers, but instruction order is still the biggest win. .dockerignore keeps the context small so COPY invalidation isn't worse than it needs to be.
COPY package.json package-lock.json ./ RUN npm ci COPY . . RUN npm run build # also: echo node_modules >> .dockerignore
Interviewer often follows with: Where would you put a secret token needed only at build time?
Link to this questionTags are mutable pointers. nginx:1.30 today can resolve to a different digest tomorrow. I'd pin by digest in production so pulls are content-addressed and reproducible.
Registries allow retagging. A digest identifies the exact image config and layers. Production manifests, Helm values, and admission policies should prefer digests. Floating tags are fine for local experimentation; they're a supply-chain and reproducibility risk in shared environments.
docker pull nginx:1.30
docker inspect --format '{{index .RepoDigests 0}}' nginx:1.30
# FROM nginx@sha256:abcd…Interviewer often follows with: How would you enforce digest pins at deploy time?
Link to this questionNamespaces decide what the process can see — PID, net, mount, UTS, IPC, user. Cgroups decide what it can consume — CPU, memory, I/O, PIDs. Together they are the container; they aren't a VM-strength security boundary.
PID namespace gives the container its own PID 1. Net namespace gives a private network stack unless you pass --network=host. Mount namespace isolates the filesystem view, including the OverlayFS merge. User namespace remaps container UIDs to unprivileged host UIDs; userns-remap uses it with a root daemon, and rootless mode also runs the daemon unprivileged. Cgroups v2 enforces memory.max and cpu.max, which is what Kubernetes and modern Docker use for limits. A kernel exploit, docker.sock mount, or --privileged still bypasses this model — hence seccomp, AppArmor/SELinux, dropped caps, and optional sandboxes like gVisor or Kata when you need higher assurance.
docker inspect web --format 'Pid={{.State.Pid}} Caps={{json .HostConfig.CapAdd}} {{json .HostConfig.CapDrop}}'
lsns -t pid,net,mnt | head
cat /sys/fs/cgroup/system.slice/docker-$(docker inspect -f '{{.Id}}' web).scope/memory.max 2>/dev/null || trueInterviewer often follows with: What does --privileged actually disable?
Link to this questionImages & builds
docker build — or buildx — reads the Dockerfile and tags an image; docker run starts a container from it. I'd map ports with -p and pass config with -e or env files.
docker build -t app:1 . docker run -d -p 8080:8080 -e LOG_LEVEL=info --name app app:1 docker logs -f appLink to this question
ENTRYPOINT sets the fixed executable. CMD supplies default arguments — or the full command if there's no ENTRYPOINT. I'd prefer exec-form JSON arrays so the process gets signals as PID 1.
ENTRYPOINT ["python", "app.py"] CMD ["--port", "8080"] # docker run img --port 9090 → python app.py --port 9090Link to this question
Multiple FROM stages in one Dockerfile. A later stage COPY --from an earlier stage takes only the artifact, so compilers and build caches never ship in the final image. That's how I'd keep runtime images thin.
FROM golang:1.27 AS build WORKDIR /src COPY . . RUN CGO_ENABLED=0 go build -o /out/app ./cmd FROM gcr.io/distroless/static COPY --from=build /out/app /app ENTRYPOINT ["/app"]Link to this question
I'd start with a small base — distroless, alpine, or scratch — use multi-stage so build tools stay out, clean package caches in the same RUN that installs, and keep a tight .dockerignore.
FROM gcr.io/distroless/base-debian12 COPY --from=build /app /app USER nonroot # + .dockerignore: .git, node_modules, tests, docsLink to this question
Parallel stages, better caching including remote cache, and mounts for secrets, SSH, and cache that never become image layers when you use them right.
BuildKit runs independent stages concurrently, supports remote cache export/import for CI, and gives you mount types — secret, ssh, cache — that keep credentials and package caches out of image layers. The legacy builder serialized steps and made it too easy to leak secrets via ARG/ENV. BuildKit has been the default builder since Engine 23.0, so DOCKER_BUILDKIT=1 only matters on old engines; I'd pin # syntax=docker/dockerfile:1. Cache invalidation from instruction order still applies on top of BuildKit features.
# syntax=docker/dockerfile:1
RUN --mount=type=cache,target=/root/.cache/pip \
pip install -r requirements.txt
RUN --mount=type=secret,id=pypi \
PIP_INDEX_URL="https://$(cat /run/secrets/pypi)@pypi.example/simple" \
pip install internal-pkgInterviewer often follows with: How do you share BuildKit cache across ephemeral CI runners?
Link to this questionI'd use them to drop shell and package managers — smaller attack surface, fewer CVEs. Debugging needs an ephemeral debug container or Docker Desktop's docker debug, not docker exec into a shell that isn't there.
Distroless and static images ship only the runtime libs your binary needs. That removes apt, bash, and curl from production, which is great for security reviews, but on-call workflows that assumed shell access have to change. I'd COPY a statically linked binary into scratch or distroless, run as nonroot, and keep a debug image or kubectl debug path documented. Dynamic linking and CA certs are the usual footguns — I'd test the final stage early in CI.
FROM gcr.io/distroless/base-debian12 COPY --from=build /app /app USER nonroot:nonroot ENTRYPOINT ["/app"]
Interviewer often follows with: How do you get a shell next to a distroless container in production without baking one in?
Link to this questionI'd use BuildKit with remote cache, secret mounts for private deps, SBOM plus CVE scan on the final digest, then cosign sign. Promote by digest, never by floating tag alone.
Pipeline shape I'd want: buildx with --cache-from/--cache-to in registry mode, Dockerfile secret mounts for npm or pip tokens, multi-stage final image, trivy or grype fail on fixable HIGH/CRITICAL for the production digest, cosign sign — keyless OIDC in CI is common — and admission verify at deploy. Cache mounts speed installs without committing caches into layers. Separate builder and runtime images so compilers never reach the cluster. Attestations bind to the digest you actually deploy.
Admission should reject unsigned or wrong-identity digests even if the tag looks familiar.
docker buildx build --secret id=npm,src=$NPM_TOKEN_FILE \ --cache-to type=registry,ref=reg/app:cache,mode=max \ -t reg/app:1 --push . trivy image --severity HIGH,CRITICAL --exit-code 1 reg/app:1 cosign sign reg/app@sha256:…
Interviewer often follows with: Where should verification happen — CI only, or also at admission?
Link to this questionThey are in-toto metadata attached to the image index. An SBOM lists the software in the image or used to build it; provenance records how it was built. BuildKit adds min-level provenance by default, and in CI I would turn on --sbom=true and --provenance=mode=max.
Attestations are stored as extra manifests in the image index, so tools can read them without pulling layers, and they travel with the digest you promote. mode=max adds more build detail than the min default. Unsigned attestations describe the build but do not prove who produced it, so I would still sign the digest with cosign and verify at admission. The classic image store cannot hold images with attestations locally, which is one reason the containerd image store matters.
docker buildx build --sbom=true --provenance=mode=max \ -t registry.example/app:1 --push . docker buildx imagetools inspect registry.example/app:1
Interviewer often follows with: Why is a provenance attestation not a substitute for signing the digest?
Link to this questiondocker history shows per-instruction size; dive visualizes wasted space and duplicated files. I'd fix by combining cleanup with install RUNs and dropping build tools via multi-stage.
docker history --no-trunc app:1
docker image inspect app:1 --format '{{.Size}}'
# dive app:1Link to this questionShell-form runs via /bin/sh -c, so your app isn't PID 1 and may not receive SIGTERM. Exec-form runs the binary directly so stop signals actually reach the process.
ENTRYPOINT ["/app"] # good — PID 1 is /app # ENTRYPOINT /app # shell form — signal handling surprisesLink to this question
Runtime & networking
A bind mount maps a host path — handy for local dev. A named volume is Docker-managed storage that survives container removal, and that's usually what I'd use for databases and durable data.
docker run -v "$PWD":/app app:1 docker volume create pgdata docker run -v pgdata:/var/lib/postgresql/data postgres:16Link to this question
-p 8080:80 programs host DNAT from host:8080 to the container IP:80. Without publish, the container is only reachable on its Docker networks.
docker run -d -p 8080:80 --name web nginx:1.30 docker port web curl -I localhost:8080Link to this question
bridge is the default single-host network; host shares the host net namespace — no isolation; none disables networking; overlay spans multiple hosts in Swarm. I'd always prefer a user-defined bridge for name DNS.
docker network create appnet docker run -d --name db --network appnet postgres:16 docker run --rm --network appnet alpine ping -c1 dbLink to this question
On a user-defined bridge, Docker's embedded DNS at 127.0.0.11 maps container or service names to current IPs. The legacy default bridge doesn't — create your own network.
User-defined bridges run an embedded DNS at 127.0.0.11 inside containers. Names map to the container's current IP, so recreate doesn't break clients that use DNS names. The default bridge lacks this and pushed people toward --link. Compose project networks inherit the same behavior using service names. Debugging "name works on my laptop" often means one side still uses the default bridge or host network mode.
docker run --rm --network appnet alpine cat /etc/resolv.conf # nameserver 127.0.0.11
Interviewer often follows with: What changes if one service sets network_mode: host?
Link to this questionIt runs a command on a schedule and marks the container healthy or unhealthy. Compose can gate on service_healthy, and operators can too — it's app readiness, not just "process exists."
HEALTHCHECK --interval=10s --timeout=2s --retries=3 \
CMD curl -fsS http://127.0.0.1:8080/health || exit 1
docker inspect --format '{{.State.Health.Status}}' appLink to this question--memory and --cpus map to cgroup limits. Without them a container can starve the host. I'd pair memory limits with app heap settings so the process actually respects the cgroup.
docker run --memory=256m --cpus=0.5 --name app app:1 docker stats appLink to this question
Compose creates a user-defined network with DNS names from service names. Hardcoded IPs, links, or assumptions about docker0 break. I'd use service DNS names and published ports intentionally.
Default bridge is a special snowflake: no automatic DNS between containers, historically used with --link. Compose project networks give stable DNS — db, redis — and isolation from unrelated containers. Failures are usually connecting to localhost instead of the service name, wrong port — container vs published host — or network_mode: host surprises. Fix by attaching both services to the same network and using the service hostname.
services:
api:
networks: [appnet]
db:
networks: [appnet]
networks:
appnet:Interviewer often follows with: When is network_mode: host justified, and what do you lose?
Link to this questiondockerd allocates a container IP on the bridge, installs DNAT/MASQUERADE rules, and conntrack forwards host:443 to containerIP:8443. Hairpin and firewalld interactions are common footguns.
With bridge networking, the container has a veth pair into docker0 or br-xxxx. Publishing a port installs PREROUTING DNAT and often POSTROUTING MASQUERADE so replies route correctly. Rootless Docker uses a userspace proxy differently and can't bind low ports without helpers. Conflicts arise when host firewalls drop forwarded packets, or when another process already binds the host port. docker port, ss -tulpn, and nft list ruleset locate the programming. Host network mode skips this path entirely — the process binds on the host stack directly.
docker port web ss -tulpn | grep ':443' nft list ruleset 2>/dev/null | head
Interviewer often follows with: How does this differ under --network=host?
Link to this questionI'd use --restart unless-stopped so the container comes back across daemon restarts, or on-failure:N to cap retries for a crash loop. The on-failure policy doesn't restart a container when the daemon restarts. For real fleets I'd still prefer an orchestrator; restart policies are a host-local safety net.
docker run -d --restart unless-stopped --name api app:1 docker update --restart on-failure:5 apiLink to this question
Security & isolation
Container UID 0 is still a privileged identity inside the mount namespace and maps toward host root in the common non-userns setup. A breakout or docker.sock mount is far worse as root. I'd set USER and run with --user.
RUN useradd -u 10001 -r app USER 10001:10001 # or: docker run --user 10001:10001 app:1Link to this question
Capabilities split root powers into units — NET_BIND_SERVICE, SYS_ADMIN, and so on. I'd drop ALL and add back only what I need instead of reaching for --privileged.
Docker starts containers with a default capability set — not empty, not full root. --privileged roughly grants all and weakens confinement. Best practice for me is --cap-drop ALL then add back only what the binary needs — often nothing, sometimes NET_BIND_SERVICE. CAP_SYS_ADMIN is a red flag. Combine with non-root USER and no-new-privileges so setuid binaries can't regain privilege. Orchestrators express the same idea in securityContext.capabilities.
docker run --cap-drop ALL --cap-add NET_BIND_SERVICE \ --user 10001:10001 -p 80:80 app:1
Interviewer often follows with: Why is CAP_SYS_ADMIN treated almost like full root?
Link to this question--read-only makes the merged rootfs immutable. The app has to write only to mounted tmpfs or volumes. That blocks a lot of malware persistence tricks and accidental layer growth.
docker run --read-only --tmpfs /tmp --tmpfs /var/run \ -v appdata:/data --user 10001:10001 app:1Link to this question
Non-root USER, --cap-drop ALL with minimal adds, --read-only, no privileged, no host namespaces, seccomp default or tighter, scan and sign the image, and keep secrets out of layers. That's the checklist I'd walk through.
Defense in depth stacks for me: filesystem — read-only plus explicit volumes; identity — non-root, optional user namespace or rootless; capability bounding set; syscall filter via seccomp; MAC via AppArmor or SELinux; network exposure minimized; and supply-chain controls — pin digest, scan, cosign. Orchestrators add PSS restricted and drop Linux capabilities in the SecurityContext. I wouldn't bake SSH or shells into runtime images. Healthchecks shouldn't require curl if a tiny native probe binary will do.
docker run --read-only --cap-drop ALL --security-opt no-new-privileges:true \ --user 10001:10001 --memory=256m --cpus=0.5 \ -p 8080:8080 app@sha256:…
Interviewer often follows with: What does no-new-privileges prevent?
Link to this questionPrivileged or SYS_ADMIN, docker.sock, hostPath or host PID/net, vulnerable mounts, and kernel exploits. Prevent with least privilege; detect with runtime policies on sensitive syscalls and unexpected mounts.
Escape is usually misconfiguration, not a 0-day: privileged containers can remount and pivot; docker.sock is API-level host root; hostPID lets you reach host processes; writable host mounts let you overwrite binaries or cron. Kernel CVEs matter when tenants are hostile. Detection: Falco or Tetragon rules for shell in a container, container drift, mount of sensitive paths, unexpected privileged starts. Prevention beats detection — PSS restricted, dropped caps, read-only rootfs, no sock mounts. For strong isolation I'd use gVisor, Kata, or separate nodes rather than trusting namespaces alone.
Alerts on suspicious syscalls after prevention controls — not a substitute for dropping privileges.
docker inspect app --format 'Priv={{.HostConfig.Privileged}} PidMode={{.HostConfig.PidMode}} Binds={{json .HostConfig.Binds}}'
# cluster: kubectl get pods -A -o json | jq '.items[] | select(.spec.containers[].securityContext.privileged==true)'Interviewer often follows with: How would you respond if Falco alerts on a shell in a distroless workload?
Link to this questionDaemon and containers run as an unprivileged user via user namespaces. A breakout lands as that user, not host root — the biggest single risk reduction vs a root dockerd, with some networking and storage limits.
Rootless maps container UIDs into a subordinate UID range owned by the user. Binding ports below 1024 needs tricks; some storage drivers differ; VPN and network paths can be more complex. It's excellent for developer machines and constrained CI. For multi-tenant production, I'd pair it with orchestrator isolation or dedicated nodes. Rootless isn't the same as USER nonroot inside a rootful daemon — both are valuable and solve different layers.
docker context ls # dockerd-rootless setup tool configures systemd --user units docker info | grep -i rootless
Interviewer often follows with: Does USER nonroot inside a rootful daemon give the same breakout safety as rootless?
Link to this questionseccomp filters syscalls — Docker's default profile already blocks many dangerous ones. AppArmor or SELinux add MAC over files and actions. Together they shrink what a compromised process can do.
Unconfined or --privileged effectively disables these guards. Custom seccomp profiles can allowlist only what the app needs, but they require syscall inventory and break when libraries change. I'd start with the Docker or runtime default, never unconfined in production, and add MAC profiles for high-value services. In Kubernetes, seccompProfile and AppArmor profiles are first-class SecurityContext fields under restricted PSS expectations.
docker run --security-opt seccomp=profile.json app:1 docker run --security-opt apparmor=docker-default app:1 # avoid: --security-opt seccomp=unconfined
Interviewer often follows with: What is the difference between seccomp and capabilities?
Link to this questionI'd scan the final digest for OS and app CVEs — fail on fixable criticals — and sign with cosign so deployers verify provenance before run.
trivy image --severity HIGH,CRITICAL --exit-code 1 registry/app@sha256:... cosign sign --yes registry/app@sha256:... cosign verify --certificate-identity-regexp 'https://github.com/org/app/.github/workflows/.*' \ --certificate-oidc-issuer https://token.actions.githubusercontent.com \ registry/app@sha256:...Link to this question
Nearly all capabilities, almost all device access, and it disables key confinement. I'd treat it as "this container may own the host" — almost never acceptable for apps.
# instead of --privileged: docker run --cap-add NET_ADMIN --cap-drop ALL … # still review carefully # better: redesign so the container does not need host-level powerLink to this question
Real-world scenarios
BuildKit reused a poisoned or stale cached FROM layer. Someone retagged the base, or a remote cache entry never invalidated. I'd pin by digest, bust the FROM cache, and treat floating tags as untrusted in CI.
Layer cache keys on the instruction text, not on what the registry means today. FROM node:20 can stay cached for days while the registry digest moves — or a bad remote cache can serve an older blob under the same cache key. Fix path for me: pin FROM …@sha256:…, use --pull carefully, and periodically rebuild base layers. Verify the running digest against the SBOM. Prevention: digest pins, admission verify, and scheduled rebuilds of golden bases.
docker pull node:20
docker image inspect node:20 --format '{{index .RepoDigests 0}}'
# Dockerfile: FROM node:20@sha256:…
docker build --pull -t app:1 .Interviewer often follows with: How do you force CI to re-resolve a floating base without discarding all app-layer cache?
Link to this questionI'd treat it as a probable host compromise path. Quarantine the workload, pull runtime logs, remove privileged, and redesign whatever "needed" host devices with a narrower capability or a dedicated node pool.
Privileged disables most confinement and grants device access — an attacker who lands RCE can pivot to the host. Immediate: stop or replace the container, snapshot forensic evidence, rotate credentials that lived on that node, and check for unexpected privileged siblings. Remediation: drop privileges, use --device for a single device if truly required, or move the job to a VM, Kata, or gVisor boundary. Long term: admission policy denies privileged in prod; CI scans Compose and Helm for privileged: true.
Privileged + shared kernel is a classic escape enabler — remove it before hunting CVEs.
docker inspect batch --format 'Priv={{.HostConfig.Privileged}} Caps={{json .HostConfig.CapAdd}}'
docker update --restart=no batch && docker stop batch
# redeploy without --privileged; add only required --cap-add / --deviceInterviewer often follows with: What evidence would convince you the host was already escaped?
Link to this questionI'd run docker system df, then hunt dangling images, unused build cache, container logs, and fat writable layers. Prune carefully — I wouldn't delete named volumes without checking.
Common fill sources: dangling images after rapid CI tags, BuildKit cache, json-file logs without max-size, and containers writing into the overlay upperdir instead of a volume. docker system df -v shows reclaimable space per category. Safe reclaim: image prune and builder prune for unused; truncate huge *-json.log only after confirming rotation; fix the app to volume-mount write paths. Dangerous: docker system prune -a --volumes on a shared host. Also check inode exhaustion when many tiny layer files accumulate, deleted-but-open log files held by long-lived containers, and on hosts using the containerd image store (the Engine 29 fresh-install default) the containerd data path outside /var/lib/docker.
docker system df -v du -sh /var/lib/docker/* 2>/dev/null | sort -h | tail docker builder prune -f docker image prune -f # fix logs: daemon.json log-opts max-size/max-file
Interviewer often follows with: How do you free space without deleting a volume that still has data an offline container needs?
Link to this questionCI context differs: .dockerignore, case-sensitive filesystem, missing git-lfs files, or a different build context path. I'd reproduce with the same context tarball CI uses, not my dirty working tree.
Local macOS or Windows often hide case and line-ending issues; Linux CI doesn't. .dockerignore may omit files present locally, or developers build from a subdirectory while CI builds from repo root. Git LFS pointers vs real blobs and sparse checkouts also bite. Process for me: print docker version and buildx, show the exact build command and context, tar the context the way CI does, and compare file lists. Fix the Dockerfile/context contract rather than "works on my machine" COPY hacks.
git ls-files | sort > /tmp/tracked docker build --no-cache -t repro . # in CI: find . -type f | sort | diff -u /tmp/tracked - cat .dockerignore
Interviewer often follows with: Why can COPY . . succeed locally and still miss a needed file in CI?
The healthcheck is probably too shallow — process up, TCP open, or a static /healthz that never hits the path users hit. I'd deepen the check and separate liveness from readiness.
Classic false green: HEALTHCHECK curls localhost:8080/ and gets 200 from a static file while /api still fails on DB or auth. Or the check runs as root while the app user can't read a required file. Another trap: restart policies keep a crash-looping process "up" between failures long enough for a generous interval. Fix: readiness must exercise critical dependencies; keep liveness cheap; emit metrics on health failure. Review docker inspect State.Health.Log for failing streaks that were masked by retries.
docker inspect app --format '{{json .State.Health}}' | jq .
curl -sS -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8080/readyz
# HEALTHCHECK should hit /readyz, not /Interviewer often follows with: When is a failing dependency a readiness failure vs a liveness failure?
Link to this questionI'd assume host-equivalent access: anyone who can reach that socket can act as root on the host. Remove the mount, rotate anything on that node, and replace the sidecar with an API that doesn't need the Docker socket.
Anyone who can speak the Docker API from that container can start privileged containers, mount host paths, and escape. Response: isolate the host, inspect which containers used the sock, review docker events and auth logs, rotate CI/registry/cloud credentials that touched the node, and redeploy without the bind. Alternatives: Kubernetes API metrics, cAdvisor without sock in modern setups, or a tightly filtered socket proxy that allows only specific API calls. CI DinD asks for the same mount casually; build with rootless BuildKit or Buildah without a socket instead (kaniko is archived). I wouldn't leave docker.sock in app or prod observability compose files "just for convenience."
docker.sock access collapses container isolation to host root-equivalent control.
docker ps -q | xargs -I{} docker inspect {} --format '{{.Name}} {{json .HostConfig.Binds}}' | grep docker.sock
# remove the volume; redeploy agent without sockInterviewer often follows with: Name a safer way to collect container metrics, and one to build images in CI, without mounting docker.sock.
Link to this questionThe tag points at a fat manifest list; platforms resolved to different digests, and one arch was stale or unsigned. I'd promote and verify by platform digest, not by tag alone.
Multi-arch images are an index of per-platform manifests. CI that only builds and scans linux/amd64 can green-light a tag while linux/arm64 still points at an old or empty blob. Symptoms: arch-specific bugs, missing shared libraries, or cosign verifying the index while a child digest was never attested. Fix: buildx --platform for both arches with a single push, scan each platform digest, sign the index and/or attestations, and pin deploy digests per node pool. crane or docker buildx imagetools inspect shows the map.
Tag → index → platform digest; verify the leaf that actually executes.
docker buildx imagetools inspect registry.example/app:1.4 crane digest --platform linux/arm64 registry.example/app:1.4 cosign verify --certificate-identity-regexp 'https://github.com/org/app/.github/workflows/.*' \ --certificate-oidc-issuer https://token.actions.githubusercontent.com \ registry.example/app@sha256:… # the arm64 child digest
Interviewer often follows with: Should admission verify the manifest list digest or the platform child digest?
Link to this questionI'd rotate the key immediately, pull and quarantine every image that ever contained it, rebuild with BuildKit secret mounts, and rewrite or delete contaminated tags from the registry. Deleting the file in a later layer never helps, because the earlier layer still holds it.
Layer history is permanent for that digest — deleting a later ENV or RUN doesn't erase earlier layers. docker history, dive, or extracting the layer tarballs recovers ARG/ENV values and copied files, and multi-stage doesn't help if the secret was copied into the final stage. Response: rotate/disable the credential in IAM, search the registry for digests whose history matches, quarantine them, force clients onto clean digests, and audit CloudTrail for use of the leaked key. Prevention: RUN --mount=type=secret for build-time credentials, runtime injection from a secret store, never ARG/ENV for credentials, multi-stage so build tools never enter the final image, and registry scanning for high-entropy strings. Assume any pulled copy outside your registry still exists.
docker history --no-trunc app:leaky | head # Dockerfile: RUN --mount=type=secret,id=aws AWS_SHARED_CREDENTIALS_FILE=/run/secrets/aws ./fetch-deps.sh docker buildx build --secret id=aws,src=$HOME/.aws/credentials -t app:clean --push . # invalidate old tags; rotate IAM key; grep CloudTrail for AccessKeyId
Interviewer often follows with: Why is squashing layers not a reliable fix for a secret that already shipped?
Link to this questionI'd treat it as compromise until proven otherwise. Isolate the workload, capture the container filesystem and events, and redeploy from a known-good digest — distroless shouldn't have a shell.
Distroless and scratch images lack a shell; a shell process means either the wrong image ran, someone exec'd a debug sidecar into the wrong namespace, or an attacker dropped a binary. Triage: compare image digest to the signed release, docker top or crictl ps, check mounts and capabilities, dump the upperdir before destroy if forensics need it, and rotate secrets the pod could reach. I'd only suppress after confirming a sanctioned ephemeral debug pattern — never globally silence shell rules in prod.
Unexpected exec in a minimal image is a high-signal compromise indicator.
docker inspect app --format '{{index .RepoDigests 0}} {{.Config.Image}}'
docker top app
# cordon node / pause traffic; snapshot; redeploy pinned digestInterviewer often follows with: How do you distinguish a legitimate kubectl debug session from malware?
I'd stop using host networking for apps that don't need it. Move them to bridge or user-defined networks with published ports, and reserve host-net only for agents that truly require the host namespace.
Host network mode removes network namespace isolation — every container shares the host's IP and port table, so two services binding :8080 collide after reboot ordering changes. It also weakens network policy and makes firewall reasoning harder. Fix: default to bridge, publish only required ports. If an agent must use host net — some CNI/eBPF tools — pin it to a dedicated node and document the port contract. I'd audit Compose for network_mode: host.
docker ps --format '{{.Names}} {{.Networks}}'
docker inspect api --format '{{.HostConfig.NetworkMode}}'
# redeploy on a user-defined bridge with -p 8080:8080Interviewer often follows with: What security properties do you lose with --network=host besides port conflicts?
I'd truncate or rotate the active log file to recover disk, restart if needed, then set daemon-wide max-size/max-file — or a better log driver — and fix the app's log spam.
Default json-file logging can grow without bound per container. Short term: identify the huge *-json.log under /var/lib/docker/containers/<id>/, truncate safely, and free space. Medium term: configure log-opts max-size and max-file in daemon.json or per-container, or ship to journald/fluentd. Root-cause the log storm — tight error loop, debug left on. Orchestrators should set container log rotation at the runtime too — host Docker config alone may not cover containerd/CRI paths.
du -ah /var/lib/docker/containers | sort -h | tail
# truncate the offender's *-json.log after noting the container id
# daemon.json: "log-driver":"json-file", "log-opts":{"max-size":"50m","max-file":"3"}Interviewer often follows with: Why might truncating the log file while the container is running be unsafe on some setups?
Link to this questionI'd keep both colors pinned by digest behind the LB, drain connections before killing green's predecessor, and only retire the old digest after health and error budgets clear — never retag in place mid-cutover.
Retagging :prod to a new digest while old tasks still run creates ambiguity for pull-on-restart and forensics. Better: run app@sha256:old and app@sha256:new as separate services, shift traffic gradually, watch golden signals, then decommission old. Pair with cosign verify on both digests at deploy time. Rollback means pointing the LB back to the previous digest still present — which fails if you already garbage-collected it. Retention policy must keep N previous production digests.
Traffic moves between digests; tags are labels, not the deployment unit.
docker run -d --name api-green app@sha256:NEW… docker run -d --name api-blue app@sha256:OLD… # LB weight blue→green; on fail, weight back; only then docker stop api-blue
Interviewer often follows with: How long do you retain the previous production digest, and who is allowed to garbage-collect it?
Link to this questionRelated
- Cheat sheetDocker CLI cheat sheet
- ComparisonDocker vs Podman
- CourseDocker for beginners
- CourseDocker in depth
- CourseAdvanced container security
- Field noteMulti-stage Docker builds that cut image size by 80%
- Field noteRootless Docker and why containers shouldn't run as root
- Field noteDistroless container images: shipping without a shell
Primary references
Found a technical issue on this page? Report it with the tool version you used and the behavior you saw. How resources are maintained.