PersistentVolumes & claims

Decoupling a storage request from the storage.

A container's filesystem is a scratch pad. Kill the container and everything it wrote vanishes with it. That's fine for a stateless web server. It's a disaster for a database. So Kubernetes needs a way to hand a Pod (the smallest thing Kubernetes runs, usually a single container) storage that outlives it, and it has to do that without nailing the Pod to one specific disk on one specific machine.

Think of the coat check at a theater. You hand over your coat and get a small numbered ticket. All night you carry the ticket, not the coat, and on the way out the ticket gets you the exact same coat back. Kubernetes splits storage the same way. The PersistentVolume (PV) is the coat: the real storage sitting in the cluster. The PersistentVolumeClaim (PVC) is your ticket: a request that says 'I need this much space, with this kind of access.' Your Pod only ever holds the ticket. It never names the disk.

Two objects, and why the split earns its keep

A PersistentVolume is a real piece of storage registered with the cluster: a cloud disk, an NFS (Network File System) export, a local SSD. It carries a size, an access mode, and a reclaim policy that decides what happens to it when nobody's using it anymore. It's cluster-scoped, meaning it doesn't live inside any one namespace (the folders Kubernetes uses to keep different teams' objects apart). A PersistentVolumeClaim does live in a namespace, and it just states a need: 20 gigabytes, read-write. A control loop inside the controller manager, the volume binder, watches for new claims and marries each one to a PV that fits. Once bound, that PV belongs to that claim alone, one to one, and no other claim can grab it. The payoff is portability. Your app's manifest names a claim, so the same YAML runs on a throwaway test cluster on your laptop and on production EKS (Amazon's managed Kubernetes), and each cluster fills the claim from whatever storage it actually has. There's a second win. The claim has a life of its own. It outlives the Pods that mount it, so you can delete and rebuild the workload as often as you like, and the storage just sits there waiting to be picked back up.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: payments-data
spec:
  accessModes: [ReadWriteOnce]
  storageClassName: standard
  resources:
    requests:
      storage: 20Gi

$ kubectl apply -f pvc.yaml

persistentvolumeclaim/payments-data created

With a default StorageClass set on the cluster, you don't pre-build anything. The claim itself triggers a PV to be created on demand (dynamic provisioning, which the next lesson pulls apart). Watch it happen:

$ kubectl get pvc,pv

NAME                                  STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/payments-data   Bound    pvc-9f2c1e4a-7b3d-4c8e-a1f2-6d5b8c9e0a1f   20Gi       RWO            standard       8s
 
NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                   STORAGECLASS   AGE
persistentvolume/pvc-9f2c1e4a-7b3d-4c8e-a1f2-6d5b8c9e0a1f   20Gi       RWO            Delete           Bound    default/payments-data   standard       8s

Read that top to bottom. The claim flipped from Pending to Bound in a few seconds. Its VOLUME column now holds an auto-generated PV name, pvc- followed by the claim's UID (its unique ID). On the PV side, STATUS is Bound and CLAIM points back at default/payments-data, so the link runs both ways. Every PVC moves through Pending, then Bound (and Lost, if its PV ever disappears underneath it). Every PV moves Available, then Bound, then Released once its claim is deleted, or Failed if reclaiming the storage errored out. Those state columns are the first thing you read when storage misbehaves.

Handing the claim to a Pod

A Pod mounts a claim exactly like any other volume. The one difference is the source: you name a claimName instead of a disk. Here's a one-replica Deployment for a database. It's one replica on purpose, because this disk can attach to a single node at a time, and that detail matters in a minute.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payments
spec:
  replicas: 1
  strategy: { type: Recreate }
  selector: { matchLabels: { app: payments } }
  template:
    metadata: { labels: { app: payments } }
    spec:
      containers:
        - name: db
          image: postgres:16
          volumeMounts:
            - name: data
              mountPath: /var/lib/postgresql/data
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: payments-data

$ kubectl exec deploy/payments -- df -h /var/lib/postgresql/data

Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme1n1     20G  148M   20G   1% /var/lib/postgresql/data

Twenty gigabytes, mounted right where the container expects it. That's the proof the claim turned into a real disk. Kill this Pod and the Deployment recreates it against the same claim, on the same data. That reattach is the whole point: the workload is disposable, the storage isn't.

When the claim won't bind

One of the most common storage complaints sounds like a Pod problem and isn't. A Pod stuck in Pending, or stuck ContainerCreating, usually traces back to a claim that never bound. The scheduler refuses to place a Pod until its claims are satisfied, so always check the claim before you touch the Pod.

$ kubectl get pvc reporting-data

NAME             STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
reporting-data   Pending                                     fast           3m

$ kubectl describe pvc reporting-data

Events:
  Type     Reason              Age               From                         Message
  ----     ------              ----              ----                         -------
  Warning  ProvisioningFailed  9s (x8 over 3m)   persistentvolume-controller  storageclass.storage.k8s.io "fast" not found

There's the whole story in one line. Someone asked for a StorageClass named fast that this cluster doesn't have, a typo or a manifest lifted from somewhere else. Fix it by pointing at a class that exists (kubectl get storageclass) or by creating that one. Other Pending reasons show up in the events just as clearly: a size no PV can satisfy, or an access mode no backend offers. Sometimes it's a cloud disk quota you've already hit. And the Pod's own events point straight back at the claim, so you never have to guess where the stall is:

Events:
  Type     Reason            From               Message
  ----     ------            ----               -------
  Warning  FailedScheduling  default-scheduler  0/3 nodes are available: pod has unbound immediate PersistentVolumeClaims.

Pending PVC usually means no matching PV or StorageClass provisioner. Describe the claim.

Retain policies leave data after claim delete. That is safety and also a cleanup chore.

Access modes must match how you mount. RWO cannot attach to two nodes at once on most block storage. Describe the claim.

Try this

Create a PVC, watch it Bind (or stay Pending), and mount it in a pod. Delete the pod and confirm the claim still holds the volume.

$ kubectl apply -f pvc.yaml
 
$ kubectl get pvc,pv
 
$ kubectl exec deploy/payments -- df -h /var/lib/postgresql/data
 
$ kubectl get pvc reporting-data
 
$ kubectl describe pvc reporting-data

Takeaway

PVCs request storage; PVs are the actual volumes. Binding decouples app YAML from the storage backend details.

Related