An app writes its cache to an emptyDir volume. The app container hits a bug and crashes, and the kubelet restarts it in place. What happens to the cache?

It's still there, because emptyDir lives as long as the Pod, not the container. A container restart re-attaches the same emptyDir. Only removing the Pod from the node (delete, or reschedule to a different node) deletes it.

A teammate proposes a hostPath volume that gives an application Pod read-write access to the node's root filesystem so it can 'write logs anywhere.' Based on the lesson, what is the main problem with this?

hostPath ties the Pod to whichever node holds that path and hands the container a well-worn escape route into the host; it should be readOnly and reserved for node agents. HostPath node-locks the Pod and exposes the host, which is why the lesson keeps it readOnly and never points application data at it.

A batch Pod writes an unbounded cache into a default emptyDir. Overnight it fills the node's root disk, the node flips to DiskPressure, and Kubernetes starts evicting unrelated Pods on that node. What is the right change to contain the blast radius?

Set emptyDir.sizeLimit so the greedy Pod trips its own cap and is evicted alone, instead of dragging the node down. SizeLimit bounds the volume so the offending Pod is evicted by itself rather than pushing the whole node into DiskPressure.

Courses Kubernetes administrationVolumes & volume types

Volumes & volume types

emptyDir, hostPath, and why pods need volumes.

Intermediate10 min · lesson 38 of 65

In plain terms

A container’s own disk is a hotel-room whiteboard, wiped at checkout. A volume is a locker you attach to the pod; a PVC-backed locker keeps your things even after you switch rooms.

Kill a container and everything it wrote to disk is gone. Not corrupted, not archived somewhere for later. Gone. Every container starts with a thin writable layer stacked on top of its read-only image, and the instant the container restarts, that layer gets thrown away and rebuilt fresh from the image. That's wonderful for keeping containers clean and identical everywhere they run. It's a disaster if your app just wrote a file it needs ten seconds from now.

Think of a container's own disk as the scratch paper a temp worker gets on day one. Handy during the shift, shredded the moment they clock out. A volume is a drawer built into the desk that stays put no matter who sits down. In Kubernetes a volume is storage you define at the Pod level (a Pod is one or more containers that always run together on the same machine) and mount into whichever containers need it. Its lifetime is tied to the Pod, not to any single container inside it. Hold onto that sentence. Everything below is a variation on it.

Why 'tied to the Pod' is the whole game

Two headaches disappear once storage outlives the container. A crash no longer wipes your data, because the kubelet (the Kubernetes agent running on every node) restarts the container in place and re-attaches the same volume. And containers in the same Pod can pass files to each other through a shared volume: a sidecar tailing the app's log file, an init container fetching data the app reads at startup. They mount one volume at different paths and see the exact same bytes.

emptyDir: shared scratch space

The simplest volume there is. An emptyDir is created empty when the Pod lands on a node and deleted when the Pod leaves that node. Good for a cache, temp files mid-computation, or handing data from one container to another. Here two containers in one Pod share a single emptyDir, each mounting it at a different path.

shared-scratch.yaml

apiVersion: v1
kind: Pod
metadata:
  name: shared-scratch
spec:
  containers:
  - name: app
    image: busybox:1.36
    command: ["sh", "-c", "sleep 3600"]
    volumeMounts:
    - name: scratch
      mountPath: /data
  - name: sidecar
    image: busybox:1.36
    command: ["sh", "-c", "sleep 3600"]
    volumeMounts:
    - name: scratch
      mountPath: /shared
  volumes:
  - name: scratch
    emptyDir: {}

Apply it, then describe the Pod. The Mounts and Volumes sections show you exactly what the kubelet wired up: the same volume mounted into both containers at two different paths.

kubectl describe pod shared-scratch

Containers:
  app:
    Mounts:
      /data from scratch (rw)
  sidecar:
    Mounts:
      /shared from scratch (rw)
Volumes:
  scratch:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>

Now prove the sharing is real. Write a file from the app container and read it back from the sidecar.

kubectl exec: write from app, read from sidecar

kubectl exec shared-scratch -c app -- sh -c 'echo "hi from app" > /data/note.txt'
kubectl exec shared-scratch -c sidecar -- cat /shared/note.txt

output

hi from app

No storage system, no provisioning, nothing to clean up afterward. Delete the Pod and the directory vanishes with it. You can also put an emptyDir in RAM by setting medium: Memory, which gives you a tmpfs (a filesystem that lives in memory instead of on disk). It's fast and wiped on reboot, but every byte it holds counts against the Pod's memory budget.

hostPath: borrowing the node's own disk

Sometimes you really do need a file from the machine itself: reading /var/log, reaching the container runtime socket, a monitoring agent that inspects the host. hostPath mounts a path from the node's filesystem straight into the Pod. The type field tells the kubelet what to expect (Directory means it must already exist, DirectoryOrCreate makes it if it's missing).

node-log-reader.yaml

apiVersion: v1
kind: Pod
metadata:
  name: node-log-reader
spec:
  containers:
  - name: reader
    image: busybox:1.36
    command: ["sh", "-c", "tail -f /host/log/syslog"]
    volumeMounts:
    - name: varlog
      mountPath: /host/log
      readOnly: true
  volumes:
  - name: varlog
    hostPath:
      path: /var/log
      type: Directory

kubectl get pod node-log-reader -o wide

NAME              READY   STATUS    RESTARTS   AGE   IP           NODE            NOMINATED NODE
node-log-reader   1/1     Running   0          9s    10.244.2.7   worker-node-2   <none>

That NODE column is the catch, and it shows up on the exam and in real incidents. hostPath nails the Pod to whichever node holds that path, so if the Pod reschedules elsewhere the data simply isn't there. And it hands a container a door straight into the host: mount the node's root filesystem or the kubelet's credentials and one compromised container owns the machine. It's a well-worn container-escape route. Keep it readOnly, reserve it for node agents that genuinely need host access, and never point application data at it.

configMap and secret volumes

Config and secrets can arrive as files too. A configMap volume turns each key into a file under your mount path. A secret volume does the same for sensitive values and keeps them in tmpfs so they never land on node disk. Your app just reads /etc/config/settings and stays blind to where it came from, which is how you ship config without baking it into the image. One trap worth knowing: mount a single key with subPath and that file stops receiving live updates when you edit the configMap, because subPath breaks the symlink Kubernetes uses to swap in new content.

Two fields on these volumes are worth setting before they bite you. defaultMode is the permission Kubernetes stamps on every file it writes, given in octal. Leave it out and you get 0644, readable by everyone inside the container, which is how a private key ends up failing a hardening check. The second is availability: a configMap or secret volume is required by default, so if the object you named does not exist yet the kubelet cannot mount it and the Pod sits in ContainerCreating until someone creates it. Set optional: true when the file is a nice-to-have, and the Pod starts anyway with the mount directory simply empty.

app-pod.yaml (the volumes section)

  volumes:
  - name: config
    configMap:
      name: app-config
      defaultMode: 0400   # owner read-only; leave it out and you get 0644
      optional: true      # missing configMap? start anyway, mount an empty dir

Diagram

Need to store data in a Pod?

pick by lifetime and source

Temp or shared between containers, fine to lose

emptyDir

dies with the Pod

Config or secret values, as files

configMap / secret volume

projected, usually read-only

A node agent needs host files

hostPath (readOnly)

node-locked, escape risk

Must survive Pod deletion and follow the app

PersistentVolume + PVC

covered next lesson

What the kubelet actually does, and how to check it

The kubelet does the physical wiring. When a Pod is assigned to a node, the kubelet stages most volumes on the node's disk first, under /var/lib/kubelet/pods/<pod-uid>/volumes/, with one subdirectory per volume type: an empty directory for emptyDir (kubernetes.io~empty-dir), written files for configMap and secret (kubernetes.io~configmap, kubernetes.io~secret). hostPath is the exception, because there is nothing to stage. The path you named on the node is used exactly as it is, so don't go hunting for a hostPath entry under that directory, you won't find one. The kubelet then hands the finished list of source paths to the container runtime, which bind-mounts each one into the container at the path you chose. When a mount misbehaves, two moves cover almost everything: kubectl describe pod to read the Mounts and Volumes sections you saw above, and kubectl exec into the container to list the actual files. If a file is missing, it's nearly always a name mismatch between volumeMounts and volumes, or a configMap that doesn't exist yet.

emptyDir shares the node's disk, and can take the whole node down

By default an emptyDir lives on the node's root filesystem, the same disk the kubelet and every other pod depend on. An app that writes gigabytes to it (a runaway log, an unbounded cache) can fill that disk. The node then flips into DiskPressure and starts evicting pods, including innocent neighbors that did nothing wrong. Set emptyDir.sizeLimit so a greedy pod trips its own cap and gets evicted alone instead of dragging the node down. When it trips, you see the eviction event below. medium: Memory is a different failure, not the same one on RAM. A memory-backed emptyDir is charged to the Pod's own memory accounting, so a Pod that already has a memory limit is partly fenced in and tends to get OOM-killed rather than evicted for node DiskPressure. Current Kubernetes also sizes that tmpfs up front, so a write past the cap usually fails on the spot with 'No space left on device' instead of triggering an eviction at all.

kubectl get events --field-selector reason=Evicted

LAST SEEN   TYPE      REASON    OBJECT           MESSAGE
12s         Warning   Evicted   pod/cache-hog    Usage of EmptyDir volume "cache" exceeds the limit "512Mi".

Try this

Go and see the staging directory for yourself on a lab cluster where you can get a shell on the node (minikube ssh, or docker exec -it <kind-node> bash). Ask the API for the Pod's UID, then list that Pod's volumes directory on the node. You get one entry for the emptyDir and one for the projected service account token every Pod gets. Add a hostPath volume to the Pod and look again: still no entry for it, because the host path is mounted straight in.

terminal

$ kubectl get pod shared-scratch -o jsonpath='{.metadata.uid}'
9c1b0a4e-2f77-4a9d-9f0b-3d2c5e6a1b88
 
# now on the node itself
$ sudo ls /var/lib/kubelet/pods/9c1b0a4e-2f77-4a9d-9f0b-3d2c5e6a1b88/volumes/
kubernetes.io~empty-dir  kubernetes.io~projected

Takeaway

Volumes give pods files that outlive a container process. emptyDir dies with the pod; hostPath ties you to a node and its security risks.

Gateway APIPersistentVolumes & claims

What Kubernetes isKubernetes fundamentals Security domains & threat modelKubernetes security & hardening The cluster attack surfaceKubernetes attack & defense Kubernetes interview questionsInterview prep