You set anonymous-auth=false on every kubelet but left authorization-mode=AlwaysAllow. What is the residual risk?

Any caller with any valid token, like a leaked service-account token, is authorized to exec into every pod on the node. AlwaysAllow skips authorization entirely, so one leaked token turns into full node access. You need authorization-mode=Webhook, not AlwaysAllow.

You switch a kubelet to authorization-mode: Webhook but leave anonymous access on. Why does that accidentally give you a detection signal?

For every request the kubelet asks the API server to approve it, and that check is a SubjectAccessReview the API server writes to the audit log. Webhook mode makes the kubelet phone home rather than decide alone, so an anonymous knock becomes a denied SubjectAccessReview on nodes/proxy that you can alert on.

You set anonymous enabled: false, authorization mode: Webhook and readOnlyPort: 0, restart the kubelet, and confirm a probe of :10250 now answers Unauthorized. A colleague still pulls your full pod inventory off that node. What went wrong?

The legacy read-only port 10255 is still serving /pods over plain HTTP, because readOnlyPort was set on a node whose kubelet reads a different config path. 10255 is a separate listener that needs no credentials. Hardening :10250 does nothing for it, so confirm nothing answers on :10255 too.

Courses Kubernetes attack & defenseNode & kubelet security

Node & kubelet security

Pod→node blast radius, isolation, metadata blocking.

Advanced30 min · lesson 11 of 15

Every node in your cluster runs a small background agent called the kubelet. It's the thing that actually starts and stops your containers: the local foreman on each machine, taking orders from the control plane. Most people think of it as a one-way street, the control plane telling the node what to do and the node quietly obeying. It isn't. The kubelet also opens a network port, 10250, and speaks its own small API (Application Programming Interface, the set of URLs a program exposes so other software can call it) over that port. Think of a big office building with a service hatch on every floor. Staff use each hatch to reach the wiring and machinery behind the wall, and it's supposed to stay locked. On a surprising number of buildings the hatch swings open with no key at all, and whoever finds it can climb inside and touch everything on that floor.

A management port on every node

Port 10250 isn't a harmless metrics endpoint you can wave off. It exposes the real controls: it can list every pod on this node, pull their logs, and run commands inside any of them. In a healthy cluster the API server is the only caller allowed near it, and every request gets authenticated first and then authorized. Two kubelet settings decide whether that promise holds. The first, anonymous-auth, decides whether a caller with no credentials is accepted at all. The second, authorization-mode, decides how a request that got in gets approved. The bad pairing is anonymous-auth set to true and authorization-mode set to AlwaysAllow. That's a hatch with no lock and no guard: anyone who can reach the port is treated as fully trusted. Managed clusters and modern kubeadm turn this off for you. Hand-rolled clusters, older installs, and config files that drifted over time get it wrong all the time.

The probe, and the exec that follows

An attacker who lands in a single pod, or who can just reach a node's IP on the network, starts with a simple question: what are you running? The /pods endpoint answers it in full. Every pod on that node, no login required.

probe the kubelet API from a pod on the network

curl -sk https://node-1:10250/pods \
  | jq -r '.items[] | "\(.metadata.namespace)/\(.metadata.name)"'

output: every pod on the node, no credentials used

kube-system/kube-proxy-8xk2p
kube-system/coredns-5d78c9f4b6-abcde
dev/build-bot-7c9f4d8-2m4nq
prod/payments-api-6b8d7c9f-lq9zt

Now the attacker knows a production payments pod is sitting right here. The kubelet has a /run endpoint that executes a command inside a named container and hands back whatever it prints. So they run one command: read that pod's service-account token off disk. A service account (SA) is the identity a pod uses when it talks to the API server, and the token is the password that proves that identity. Modern tokens are bound to the pod, so they stop working the moment the pod dies. That's still plenty of time. With the token in hand, the attacker stops being an outsider poking a port. Now they're prod/payments-api, walking up to the cluster's front door with a valid badge.

exec into the pod via the kubelet and steal its token

curl -sk -XPOST \
  "https://node-1:10250/run/prod/payments-api-6b8d7c9f-lq9zt/payments-api" \
  -d "cmd=cat /var/run/secrets/kubernetes.io/serviceaccount/token"

output: the pod's live service-account token (truncated)

eyJhbGciOiJSUzI1NiIsImtpZCI6Ik5nZ1..._Zt3QwFq9
Use it against the API server:
  kubectl --token=eyJhbGciOi... auth can-i --list

This is the part that should keep you up. That whole sequence never touched the API server, so if the only thing you watch is API audit logs, you saw nothing. An open kubelet is a blind spot by design. The good news is that the fix and the detection turn out to be the same switch. Flip authorization-mode to Webhook.

Make the kubelet a tripwire

Webhook mode changes how the kubelet makes up its mind. Instead of ruling on requests by itself, it phones home. For every request that arrives, it asks the API server one yes-or-no question: is this caller allowed to do this thing on this node? It's the guard at the hatch radioing head office to check a badge before he lets anyone through. That yes-or-no question is a real Kubernetes object called a SubjectAccessReview (SAR, an 'is this allowed?' check), and because the kubelet creates it against the API server, the API server writes it straight to the audit log. So a cluster that switched Webhook on but forgot to also turn off anonymous access ends up with an accidental tripwire. Every anonymous knock on the hatch now lands as a denied SAR you can alert on.

hunt the audit log for kubelet authorization checks

jq -c 'select(.objectRef.resource=="subjectaccessreviews")
  | {t:.requestReceivedTimestamp, by:.user.username,
     subject:.requestObject.spec.user,
     attrs:.requestObject.spec.resourceAttributes,
     allowed:.responseObject.status.allowed}' \
  /var/log/kubernetes/audit.log

output: an anonymous exec attempt, denied and recorded

{"t":"2026-07-16T09:41:22Z","by":"system:node:node-1",
 "subject":"system:anonymous",
 "attrs":{"resource":"nodes","subresource":"proxy",
          "verb":"create","name":"node-1"},
 "allowed":false}

Read that line slowly. The kubelet on node-1, identifying itself as system:node:node-1, asked whether system:anonymous was allowed to create against nodes/proxy. That path, nodes/proxy, is exactly how the kubelet describes a POST to /run: a POST becomes the verb create, and everything outside the stats and metrics paths falls under the proxy subresource. The answer came back false. An anonymous caller trying to run a command on your node just wrote its own detection into your audit log.

You probe a kubelet on :10250. What happens?

curl -sk https://node:10250/pods

an unauthenticated request arrives at the kubelet

anonymous-auth=true, authz=AlwaysAllow

Full foothold, no trace

lists and execs every pod; the API server never hears about it

anonymous-auth=false

401 at the door

rejected at authentication, before authorization even runs

anonymous still on, authz=Webhook

Rejected and logged

kubelet posts a SubjectAccessReview; the denied SAR is your alert

The audit trail only exists when the kubelet consults the API server. AlwaysAllow answers locally and silently, which is why it's the truly dangerous one.

Lock it down

Detection is the tripwire. It's not the lock. To actually stop the caller, you close the hole in the kubelet's own config: refuse anonymous callers, make the API server vouch for any token before you trust it, and shut off the legacy read-only port. Three fields, dropped into the config on each node, then a kubelet restart.

/var/lib/kubelet/config.yaml

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
  anonymous:
    enabled: false
  webhook:
    enabled: true
authorization:
  mode: Webhook
readOnlyPort: 0

re-run the probe after restarting the kubelet

curl -sk https://node-1:10250/pods
 
# expected:
Unauthorized

One more control finishes the job, and this one lives on the API server side. The Node authorizer hands each kubelet only the narrow slice of API access it genuinely needs. The NodeRestriction admission plugin then stops a kubelet's credentials from editing any Node or Pod object except the ones running on its own machine. So even if someone steals a kubelet's certificate off a compromised box, they can't turn it into a tool for rewriting other nodes or scheduling pods across the cluster. Make sure both are switched on.

verify Node authorizer and NodeRestriction are enabled

grep -oE 'authorization-mode=[^ "]+|enable-admission-plugins=[^ "]+' \
  /etc/kubernetes/manifests/kube-apiserver.yaml

output: both controls present

authorization-mode=Node,RBAC
enable-admission-plugins=NodeRestriction

Locking :10250 does nothing if :10255 is still open

Older and hand-provisioned clusters often left the kubelet's read-only port, 10255, serving /pods and /metrics over plain HTTP with zero authentication. You can harden 10250 perfectly and still hand out your entire pod inventory over 10255. It's a separate setting. Put readOnlyPort: 0 in the kubelet config, then confirm nothing answers on 10255 from inside the network. Check both ports, not just the famous one.

Try this

Work through “Lock it down” yourself on a sandbox you can throw away, following the commands above in order. Then break one step deliberately and re-run, so you have seen the failure before it finds you.

Takeaway

The trap worth remembering here: locking :10250 does nothing if :10255 is still open. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Control plane & etcdOperator & controller security

What Kubernetes isKubernetes fundamentals Cluster architectureKubernetes administration Security domains & threat modelKubernetes security & hardening Kubernetes interview questionsInterview prep