Debugging Pending pods: the five usual causes

Insufficient resources, taints, affinity, PVCs, and quotas — how to read the events and fix each one fast.

A pod stuck in Pending has not been scheduled to a node yet. Kubernetes almost always tells you why in the Events section of kubectl describe pod — read that before you delete anything, scale anything, or open a ticket with your cloud provider. Deleting and recreating a Pending pod does not fix insufficient CPU; it comes back Pending with the same event.

After the event message, work through five usual causes in order: resource requests exceeding node capacity, taints without tolerations, affinity rules matching zero nodes, a PVC stuck in Pending, or a namespace ResourceQuota ceiling. The Kubernetes administration course walks scheduling primitives in depth if node selectors and affinity are new to you.

Always start with describe and Events

The scheduler emits a FailedScheduling event with a human-readable reason. That string is the answer — your job is to map it to a fix, not to guess.

The five usual causes and their fixes

Insufficient cpu or memory means no node has enough allocatable capacity after subtracting existing pod requests. Taints repel pods unless a matching toleration exists. Affinity and nodeSelector shrink the candidate set — sometimes to zero. A Pending PVC blocks every pod that mounts it. ResourceQuota caps total requests per namespace.

Commands for each cause

Run these in order until the event message maps to one of them. Compare pod resource requests against node allocatable capacity, not node capacity — system and daemonset pods consume allocatable too.

# capacity and readiness
kubectl get nodes -o wide
kubectl describe node <node> | grep -A5 "Allocatable"
 
# taints vs tolerations
kubectl describe node <node> | grep -A4 Taints
kubectl get pod api-7c9f2 -n prod -o yaml | grep -A6 tolerations
 
# PVC blocking scheduling
kubectl get pvc -n prod
kubectl describe pvc data-api-7c9f2 -n prod
 
# namespace quota exhaustion
kubectl get resourcequota -n prod
kubectl describe resourcequota -n prod

PVC Pending blocks every pod that mounts it

If the pod spec references a PersistentVolumeClaim, the scheduler waits until that PVC reaches Bound. A missing StorageClass, exhausted disk on the provisioner, or a volume topology mismatch leaves the PVC in Pending and the pod with it. kubectl describe pvc shows the provisioner events — fix the storage layer, not the pod.

ResourceQuota: the silent ceiling

Quota exhaustion looks like insufficient cluster capacity but the cluster has free nodes. The namespace has hit its CPU, memory, pod count, or PVC limit. kubectl describe resourcequota shows used versus hard limits — raise the quota or delete unused objects.

Where this goes next

Pending pods are a scheduling problem. CrashLoopBackOff is a runtime problem. ImagePullBackOff is a registry problem. Each status has its own triage path. The Kubernetes administration track covers scheduling, troubleshooting, and day-2 operations end to end.

Related posts