Kubernetes autoscaling with the Horizontal Pod Autoscaler

Scale on CPU, memory, or custom Prometheus metrics, and tune stabilization windows so the HorizontalPodAutoscaler won't flap under spiky, bursty load.

The Horizontal Pod Autoscaler adds and removes replicas to match load. The manifest itself is short; the parts teams get wrong are needing metrics-server installed, setting CPU requests so utilization math works, picking a metric that actually reflects user pain, and tuning behavior so replica counts do not yo-yo under spiky traffic.

HPA v2 supports Resource metrics (CPU, memory), custom metrics from Prometheus or other adapters, and external metrics like queue depth. CPU-target autoscaling is the starting point — but for a queue worker you scale on backlog, not processor idle time. The Kubernetes administration track covers requests, limits, and scheduling before you layer autoscaling on top.

Prerequisites: metrics-server and requests

HPA reads current utilization from the metrics API. If kubectl top pods fails, HPA has nothing to scale on. Every container in the target Deployment needs a CPU request — utilization is measured as usage divided by request, not limit. A container with no request reports as 0% and the autoscaler never fires.

The basic CPU autoscaler

Target 70% average CPU utilization across pods. Set minReplicas high enough to survive a single-node loss and maxReplicas low enough to stay within your cloud budget and downstream connection limits. Leave headroom above minReplicas so the scaler has room to react before users notice latency.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api
  namespace: prod
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

Scale on a custom metric when CPU lies

CPU utilization stays flat when your bottleneck is I/O wait, thread pool exhaustion, or downstream latency. Install the Prometheus Adapter (or your cloud provider's equivalent), expose a metric like http_requests_per_second, and reference it in the HPA metrics array with type: Pods and averageValue. The scaler now reacts to traffic, not processor heat.

metrics:
  - type: Pods
    pods:
      metric:
        name: http_requests_per_second
      target:
        type: AverageValue
        averageValue: "500"

Stop the flapping with behavior policies

By default HPA scales up fast and down immediately — fine for steady load, painful for bursty traffic. A scale-down stabilization window waits before removing replicas, and a percent-based policy caps how many pods disappear per minute. Scale-up can stay aggressive; scale-down should be conservative so a brief lull does not halve your capacity right before the next spike.

behavior:
  scaleDown:
    stabilizationWindowSeconds: 300
    policies:
      - type: Percent
        value: 50
        periodSeconds: 60
  scaleUp:
    stabilizationWindowSeconds: 0
    policies:
      - type: Percent
        value: 100
        periodSeconds: 60

Where this goes next

HPA scales pods horizontally. Vertical Pod Autoscaler right-sizes requests over time. Cluster Autoscaler adds nodes when pending pods cannot fit. Together they are the three autoscalers — each solves a different bottleneck. The Kubernetes administration path covers all three with production tuning patterns.

Related posts