Horizontal Pod Autoscaler

Scaling replicas on CPU, memory, or custom metrics.

Every morning at nine, your login service buckles. Everyone clocks in at once. Traffic triples inside ten minutes, your three Pods peg their CPU, and requests start timing out. By the time you notice and bump the replica count by hand, the rush is over and you're paying for Pods nobody needs. You can't stand at the door counting customers all day. That's the job you hand to the Horizontal Pod Autoscaler.

Think of a manager working a restaurant floor. Tables fill up, she calls in extra servers. The dinner rush fades, she sends a couple home. She's holding one thing steady: enough staff for the number of guests, no more. The Horizontal Pod Autoscaler, or HPA, does that for a Deployment. A Pod is one running copy of your app. A Deployment is the controller that keeps a chosen number of those copies (its replicas) alive and healthy. The HPA watches how hard the Pods are working and nudges that replica count up or down, holding the load per Pod near a target you pick. More Pods is horizontal scaling. Giving each Pod a bigger slice of CPU would be vertical scaling, and that's a separate tool called the Vertical Pod Autoscaler.

What 60% is actually 60% of

This is where people get burned. You tell the HPA to hold CPU at 60%. Sixty percent of what, though? Not the node. Not one whole core. It's 60% of what each Pod asked for. A resource request is the slice of CPU or memory a Pod reserves when the scheduler places it, written right into the Pod spec. Say a Pod requests 200m, which is 200 millicores, roughly a fifth of one CPU core. If it's actually burning 120m, that's 60% utilization. The request is the yardstick. Take the yardstick away and the HPA has nothing to measure against, so utilization-based scaling quietly does nothing at all. No error, no scaling, just a dashboard number that never moves.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 3            # drop this line once the HPA owns the Deployment
  selector:
    matchLabels: { app: web }
  template:
    metadata:
      labels: { app: web }
    spec:
      containers:
        - name: web
          image: nginx:1.27
          resources:
            requests:
              cpu: 200m    # the yardstick utilization is measured against
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 60

kubectl apply -f hpa.yaml

deployment.apps/web created
horizontalpodautoscaler.autoscaling/web created

The loop, and the arithmetic behind it

The HPA isn't magic, and it isn't instant. Inside the control plane sits the kube-controller-manager, the process that runs Kubernetes' background reconcilers (the little loops that keep nudging reality toward what you asked for). One of those loops belongs to the HPA. It wakes up every 15 seconds by default. Each pass, it asks the metrics API for current usage, one number per Pod, then runs a single line of arithmetic: desiredReplicas = ceil(currentReplicas x currentUtilization / targetUtilization).

Work a real example. Four Pods are averaging 90% against a 60% target. Four times ninety, divided by sixty, is six, so the HPA moves you to six Pods. It always rounds up. There's a built-in tolerance baked in too: if the ratio lands within 10% of the target, roughly 54% to 66% here, the HPA leaves things alone. That's on purpose, so it isn't twitching on every tiny wobble in the numbers. CPU and memory are just the built-in options. The same v2 API can scale on custom metrics, like requests per second coming off your ingress, or external ones, like the depth of a queue in a cloud message broker. You need an adapter feeding those numbers into the metrics API, but the HPA's job never changes. Pick a number, hold it steady.

kubectl describe hpa web

Name:                          web
Reference:                     Deployment/web
Metrics:                       ( current / target )
  resource cpu on pods  (as a percentage of request):  91% (182m) / 60%
Min replicas:                  3
Max replicas:                  20
Deployment pods:               6 current / 6 desired
Conditions:
  Type            Status  Reason            Message
  ----            ------  ------            -------
  AbleToScale     True    ReadyForNewScale  recommended size matches current size
  ScalingActive   True    ValidMetricFound  the HPA calculated a replica count from cpu resource utilization
Events:
  Type    Reason             Age   Message
  ----    ------             ----  -------
  Normal  SuccessfulRescale  1m    New size: 6; reason: cpu resource utilization (percentage of request) above target

Scaling up is quick. Scaling down is slow on purpose. If the HPA ripped Pods away the instant load dipped, one quiet minute would strip your capacity right before the next spike, and you'd flap between sizes all day long. So the controller keeps a stabilization window, five minutes by default, and during that window it only ever scales down to the highest count it wanted. Think of it as a cool-down timer. If five minutes is too sleepy or too jumpy for your traffic, you can tune both directions under the behavior field.

When TARGETS shows <unknown>

The classic broken HPA looks like this. The percentage that should be a live number reads as a placeholder instead, and the replica count sits frozen at the bottom of its range:

kubectl get hpa web

NAME   REFERENCE        TARGETS              MINPODS   MAXPODS   REPLICAS   AGE
web    Deployment/web   cpu: <unknown>/60%   3         20        3          2m

That <unknown> means the HPA asked for a reading and got nothing back. With no reading it can't do the math, so it doesn't scale, and the count just sits where it launched (here, the minimum it started with). Two causes cover almost every case. Either the Pods carry no CPU request, so there's no denominator for the percentage, or the metrics-server isn't installed and can't reach the kubelets. The kubelet is the agent running on every node, and one of its jobs is reporting each Pod's live resource use. Check the metrics pipeline first, because it's the fastest thing to rule out:

kubectl top pods

NAME                   CPU(cores)   MEMORY(bytes)
web-6b9c8d4f7c-4xk2p   182m         48Mi
web-6b9c8d4f7c-7dtwq   176m         51Mi
web-6b9c8d4f7c-9r2vx   180m         49Mi

If kubectl top returns real numbers, the metrics-server is healthy and your actual problem is missing requests. Add a CPU request to the Pod spec and the percentage shows up. If instead it errors with something like 'Metrics API not available', the metrics-server itself is down or unreachable. Fix that, and the HPA picks up a reading within a loop or two, usually inside half a minute.

Custom metrics need an adapter. CPU is the tutorial path; production often needs QPS or queue depth.

thrashing happens when scale-up and scale-down are too eager. stabilizationWindow exists for a reason.

HPA and vertical sizing are different knobs. Raising requests can lower replica counts for the same load. Custom metrics need an adapter.

stabilizationWindow exists for a reason.

Try this

Deploy an app with requests set, install metrics-server if needed, create an HPA on CPU, then generate load and watch replicas climb.

$ kubectl apply -f hpa.yaml
 
$ kubectl describe hpa web
 
$ kubectl get hpa web
 
$ kubectl top pods

Takeaway

HPA scales replica count from metrics. Without requests and a metrics pipeline, it cannot think. Limits alone do not drive average utilization.

Related