>_SecOpsLog_
CoursesKubernetes attack & defenseRBAC & the powerful grants

RBAC & the powerful grants

secrets, pods create, exec — escalation in disguise.

Advanced35 min · lesson 4 of 15

Most Kubernetes cluster takeovers don't start with a zero-day. They start with a grant somebody approved on a Friday. RBAC (Role-Based Access Control) is the cluster's authorization layer, the thing that decides which identity gets to do what, and to which objects. Think of a building full of locks. Every user and every service account (SA, the identity a running program uses instead of a human login) carries a key ring. Most keys open one door. A few are master keys that quietly open the vault. A rare few are locksmith keys: they let whoever holds them cut brand-new keys for any lock in the building. Your job on defense is to know who holds which, and to find the master keys before an attacker enumerates them for you.

The grants that quietly open the vault

An RBAC rule is small. It names a set of verbs (get, list, watch, create, update, patch, delete), a set of resources they act on (pods, secrets, deployments), and it gets tied to an identity by a RoleBinding inside a single namespace or a ClusterRoleBinding across the whole cluster. That's the whole model. The danger is that a few grants read as boring on paper and behave like god mode in practice. get or list on secrets hands back every credential in that namespace in plaintext, because Kubernetes stores Secret values base64-encoded, and base64 is a costume, not a lock. Anyone can peel it off in one command. create on pods is a node-compromise primitive: a pod you're allowed to create can mount any service account's token, mount the host's own filesystem through a hostPath volume, or run privileged, and any one of those is a short walk to root on the machine. The pods/exec subresource runs commands inside a pod that's already running, inheriting that pod's token and Secrets as if you'd been sitting there all along.

Then there are three verbs that have nothing to do with reading or writing your data. They mint authority itself, and one internal rule is what makes them dangerous. The RBAC authorizer will not let you create or edit a Role that grants permissions you don't already hold. That check is called privilege-escalation prevention, and it exists so a namespace admin can't quietly write themselves a cluster-admin role. The escalate verb turns that check off, but it writes nothing by itself: the subject still needs create or update on roles, and escalate is what lets the role it writes name anything at all. bind is the sibling trick, and it needs a write verb beside it too. With create on rolebindings you attach an existing role (say cluster-admin) to yourself with a fresh binding, again without holding those powers first. Watch which resource carries the bind, because it is checked against the kind the binding points at: reaching cluster-admin takes bind on clusterroles, not bind on roles. impersonate is the third. It lets you send requests as another identity with kubectl --as, or as another group with --as-group. Point it at the group system:masters and you inherit full control, because a default binding the cluster keeps recreating hands system:masters the cluster-admin role. Two more grants do the same job from outside RBAC and belong on the same list. create on serviceaccounts/token mints a working token for any service account in that namespace without needing a pod at all, which is currently the most common way a foothold in one namespace turns into something bigger. update on certificatesigningrequests/approval, paired with approve on the matching signer, lets the holder wave through a certificate naming any user or group it likes, system:masters included. Find any of these in a workload's grants and you've found a key that cuts other keys.

Attacker and defender open with the exact same move: ask the API server, the control-plane front door every kubectl command flows through, what a given identity can actually do. The --as flag runs the question as that subject, which any cluster-admin is allowed to do.

what can this identity really do?
kubectl auth can-i --list --as=system:serviceaccount:prod:ci-runner -n prod
output
Resources                                       Non-Resource URLs   Resource Names   Verbs
*.*                                             []                  []               [*]
                                                [/api/*]            []               [get]
                                                [/healthz]          []               [get]
selfsubjectreviews.authentication.k8s.io        []                  []               [create]
selfsubjectaccessreviews.authorization.k8s.io   []                  []               [create]
selfsubjectrulesreviews.authorization.k8s.io    []                  []               [create]

That top line, *.* paired with the verb [*], means this CI (Continuous Integration) service account can do everything, to everything. It's cluster-admin wearing a dull name. Notice the -n prod on the end of that command. --list answers for one namespace at a time and falls back to whatever namespace your kubeconfig is pointed at when you leave it off, so ask about a prod service account from a default-namespace context and every Role bound to it inside prod is quietly missing from the reply. Now widen the lens to the whole cluster and find every holder of the master key.

who holds cluster-admin, cluster-wide?
kubectl get clusterrolebindings -o json \
| jq -r '.items[]
    | select(.roleRef.name=="cluster-admin")
    | .metadata.name as $b
    | (.subjects // [])[]
    | "\($b)\t\(.kind)/\(.namespace // "-")/\(.name)"'
output
cluster-admin       Group/-/system:masters
prod-ci-admin       ServiceAccount/prod/ci-runner
dashboard-full      ServiceAccount/kubernetes-dashboard/kubernetes-dashboard

The system:masters row is expected. That's the break-glass group baked into the bootstrap admin certificate, and it's supposed to hold the keys to everything. The other two rows are the problem. A CI runner and the dashboard's own service account are both wearing the master key, and neither one needs it. Next, go hunting for the locksmith keys and the wildcards, wherever they've been hiding across every Role and ClusterRole.

find escalation verbs and wildcards
kubectl get clusterroles,roles -A -o json \
| jq -r '.items[] as $r
    | select($r.metadata.name | startswith("system:") | not)
    | ($r.rules // [])[]
    | select( any(.verbs[]?;     . == "*" or . == "bind" or . == "escalate" or . == "impersonate")
           or any(.resources[]?; . == "*") )
    | "\($r.kind)/\($r.metadata.namespace // "-")/\($r.metadata.name)  verbs=\(.verbs)  res=\(.resources)"'
output
ClusterRole/-/cluster-admin        verbs=["*"]                         res=["*"]
ClusterRole/-/metrics-reader       verbs=["get","list"]                res=["*"]
Role/prod/deploy-manager           verbs=["create","bind","escalate"]  res=["roles","rolebindings","clusterroles"]

That startswith filter is doing real work. Leave it out and the sweep comes back with dozens of rows, because system:controller:generic-garbage-collector and most of its sibling controller roles hold resources ["*"] by design, and you would spend the afternoon reading grants Kubernetes shipped itself. What survives the filter is short enough to read line by line. cluster-admin holding * on * is working as designed. metrics-reader granting get and list on * looks read-only, and it isn't: resources ["*"] sweeps in secrets, so a role that sounds like harmless monitoring can read every credential in the cluster. Then deploy-manager holds create sitting next to bind and escalate, across roles, rolebindings and clusterroles. Read those verbs together, because bind and escalate alone are inert: they write nothing, they only switch off the escalation guard on a write you were already allowed to make. Here the write verb is right beside them. create plus escalate on roles lets the workload write itself a Role naming any permission it likes, and create on rolebindings plus bind on clusterroles lets it drop a binding to the cluster-admin ClusterRole straight into its own namespace. That pairing is the quiet master key, tucked inside a namespaced Role nobody thought to audit.

Found a grant. Does it hand over the cluster?
A workload or user holds a grant
Ask: what does this key really open?
secrets: get / list
Reads every credential in scope
base64 is an encoding, not a lock; decode the tokens and reuse them
pods: create / exec
A path onto the node
mount any SA token or the host filesystem, run privileged, or exec into a live pod
bind / escalate
Cuts new keys
alone they write nothing; paired with create or update, escalate rewrites a role past the guard and bind attaches an existing admin role
impersonate system:masters
Borrows an un-removable admin
a default binding the cluster keeps recreating gives that group cluster-admin
verbs or resources = ["*"]
A blank check
the wildcard quietly includes the secrets and pods you never meant to grant
The first two get an attacker moving. The last three hand over the cluster. Find them before an attacker enumerates them for you.

Detect the abuse, then close the door

You can't eyeball every grant forever, so let the cluster watch for you. The audit log is the API server's write-down of every authenticated request that reaches it, like a door-access log for the whole building. It writes nothing until somebody turns it on: a stock cluster ships no audit policy, so no file appears until the API server is started with --audit-policy-file and --audit-log-path, and that policy decides which requests get recorded and how much of each one. Check that before you go hunting for the file, and expect a managed cluster to have it on already and ship the entries to the provider logging service instead of somewhere you can grep. Two entries matter more than the rest, and both are rare in a healthy cluster: someone creating a ClusterRoleBinding, and someone impersonating the system:masters group. Loud, obvious, easy to catch.

catch RBAC self-grants and system:masters impersonation
jq -c 'select( ((.impersonatedUser.groups // []) | index("system:masters"))
            or (.verb == "create" and (.objectRef.resource | IN("clusterrolebindings","rolebindings"))) )
  | { ts: .requestReceivedTimestamp, user: .user.username,
      asGroup: ((.impersonatedUser.groups // []) | join(",")),
      verb, res: .objectRef.resource, name: .objectRef.name }' \
  /var/log/kubernetes/audit.log
output
{"ts":"2026-07-16T09:14:02Z","user":"system:serviceaccount:prod:ci-runner","asGroup":"","verb":"create","res":"clusterrolebindings","name":"ci-runner-admin"}
{"ts":"2026-07-16T09:15:40Z","user":"[email protected]","asGroup":"system:masters","verb":"delete","res":"pods","name":"audit-agent-7f9"}

Two log lines, two attacker moves. The CI runner minted itself a cluster-admin binding. Then a developer ran a delete while impersonating system:masters and wiped an audit-agent pod, which is textbook anti-forensics: kill the thing that's watching you. If you'd rather get paged than grep at 2 a.m., Falco ships a maintained k8saudit ruleset that catches the same behavior live, with rules named exactly "K8s ClusterRoleBinding Created" and "Full K8s Administrative Access". Detection tells you it happened. The fix is what stops it happening twice.

Tightening a role doesn't recall the keys already handed out
Scoping a Role down changes what new requests are allowed. It does nothing about the access already in flight. A pod that grabbed a token before your fix keeps using it until that token expires or the pod restarts, and a long-lived token an attacker already copied keeps working no matter what you edit. So after you narrow or delete a risky grant, rotate the affected service account's tokens and restart the workloads that held them. One identity you can't scope at all: anyone holding the bootstrap admin kubeconfig authenticates as the system:masters group, and a default ClusterRoleBinding hands that group cluster-admin. The cluster recreates that binding if you delete it, and RBAC only ever adds permissions, it can't subtract them, so no role you write will fence that identity in. Guard that kubeconfig like a root password.

Closing deploy-manager down means handing it exactly the verbs its job needs and not one more. It manages deployments, so grant that and nothing else, and drop create, bind, escalate and all three RBAC resources completely.

deploy-manager-scoped.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: deploy-manager
  namespace: prod
rules:
  - apiGroups: ["apps"]
    resources: ["deployments"]
    verbs: ["get", "list", "watch", "update", "patch"]
apply the fix, then prove the locksmith key is gone
kubectl apply -f deploy-manager-scoped.yaml
kubectl auth can-i create rolebindings --as=system:serviceaccount:prod:deployer -n prod
kubectl auth can-i update deployments  --as=system:serviceaccount:prod:deployer -n prod
output
role.rbac.authorization.k8s.io/deploy-manager configured
no
yes

That's the full loop on one grant: you found it, you caught it being abused in the audit log, and you scoped the role until the locksmith key was gone (no) while the deploy job kept working (yes). Put that same jq sweep on a cron schedule and new master keys surface the day they're created, not the day someone finally uses one.

Quick check
01Your jq sweep across every Role and ClusterRole returns this row: Role/prod/deploy-manager verbs=["create","bind","escalate"] res=["roles","rolebindings","clusterroles"]. A teammate says it is only a Role in one namespace, so the blast radius stops at prod. What can the identity bound to it actually reach?
Incorrect — The Role object is namespaced, but a RoleBinding created inside prod can point at a cluster-scoped ClusterRole, and bind on clusterroles is what lets that stick. The namespace does not contain it.
Incorrect — Half right. bind and escalate write nothing on their own, but the verb that wakes them up is create, sitting in the same rule. impersonate is a separate route to power, not a prerequisite for this one.
Correct — create on rolebindings supplies the write, and bind on clusterroles switches off the escalation check for the kind that binding names. The result is cluster-admin powers over prod, including every Secret in it.
Incorrect — bind is checked against the kind the binding points at, which is the right instinct. Look again at the resource list though: clusterroles is sitting right there beside roles, so cluster-admin is in reach.
02A dashboard runs with a ClusterRole granting get and list on resources ["*"], attached by a ClusterRoleBinding. An attacker steals that dashboard service account's token. Past viewing charts, what does the token hand them?
Correct — A wildcard resource covers Secrets like anything else, and Kubernetes stores Secret values base64-encoded, which is a costume rather than a lock. Read-only stops being harmless the moment the reads include credentials.
Incorrect — Exfiltration is a read. Nothing has to be modified for a credential to leave the cluster, so the verbs being read-only is exactly why this grant gets waved through.
Incorrect — A rule grants exactly the verbs it lists, here get and list, so no write or escalate arrives with the wildcard. The damage is bounded by what those two reads can reach, which is already plenty.
Incorrect — Secrets are namespaced, but this is a ClusterRole attached by a ClusterRoleBinding, so the rule applies in every namespace at once. Scope comes from the binding, not from the resource.
03You apply deploy-manager-scoped.yaml, then run kubectl auth can-i create rolebindings --as=system:serviceaccount:prod:deployer -n prod and get no, while update deployments returns yes. Your teammate calls the incident closed. What still needs doing?
Incorrect — can-i answers for a request made right now. It says nothing about a token an attacker already copied, which keeps working until it expires or the pod holding it restarts.
Incorrect — The cluster puts that default binding straight back, and RBAC only ever adds permissions, so there is nothing to subtract with. Guard the bootstrap kubeconfig like a root password instead.
Incorrect — RBAC has no deny. Rules only add, so a verb you left out of the scoped Role is already refused, and there is no denial to write.
Correct — Narrowing a Role changes what new requests are allowed and nothing else. A workload that grabbed a token under the old grant keeps using it, so rotation plus a restart is what actually recalls the access.

Every command in this lesson leaned on one thing: an identity like system:serviceaccount:prod:ci-runner. That name isn't an abstraction. It rides inside the pod as a token file on disk, and stealing that file is how an attacker gets to run --as for real, from the outside. Where those tokens come from, how they're tied to a pod's lifetime, and how to stop a pod from carrying one it never needed: that's the next lesson.

Try this

Bring up a throwaway cluster with kind or minikube and run both jq sweeps against it. The cluster-admin sweep should come back with one row, system:masters, and nothing else. Then create a service account, bind it to a Role carrying create on rolebindings and bind on clusterroles, re-run the second sweep, and watch your own planted grant appear next to cluster-admin. The audit-log query needs the API server started with --audit-policy-file and --audit-log-path before there is any file to read, so either set that up first or read that section rather than run it.

Takeaway

The trap worth remembering here: tightening a role doesn't recall the keys already handed out. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Threat-modeling a clusterService-account tokens

Related

What Kubernetes isKubernetes fundamentalsCluster architectureKubernetes administrationSecurity domains & threat modelKubernetes security & hardeningKubernetes interview questionsInterview prep