Policy at admission

Gatekeeper/Kyverno as the runtime backstop.

A nightclub with a bouncer on the front door and an unlocked fire exit round the back is not a controlled room. That is your Kubernetes cluster when policy lives only in the build pipeline. CIS (Center for Internet Security) Kubernetes Benchmark control 5.2.2, minimize the admission of privileged containers, sits inside your SOC 2 (System and Organization Controls 2) and PCI DSS (Payment Card Industry Data Security Standard) scope, so the auditor asks you something blunt: prove that no privileged Pod can run here. Your Conftest gate, the policy checker you run over manifests, already fails anything carrying privileged: true in CI (continuous integration). But CI only ever sees what flows through Git. It cannot stop a break-glass admin typing kubectl run rogue --privileged straight at the API (application programming interface) server, and it cannot see a Helm chart, a packaged bundle of Kubernetes manifests, that sets the flag three layers deep. Admission control is the lock on that back door.

Validating vs mutating admission

A passport desk has two officers. One fills in the fields you left blank on your landing card. The other reads the finished card and stamps it yes or no. Kubernetes admission works the same way, and the order matters. Every create and every update arrives at the Kubernetes API server, and before the object is written to etcd (the cluster's key-value store, the record of everything that exists), it runs through a chain of admission controllers. Two kinds of admission webhook carry policy. A validating webhook reads the incoming object and answers allow or deny; it never changes the object. A mutating webhook runs earlier and can rewrite the object before it is stored: add a default label, set runAsNonRoot: true, inject a sidecar. Gatekeeper, the Kubernetes-native distribution of OPA (Open Policy Agent), is a validating webhook driven by the same Rego, OPA's policy language, that you already write for Conftest. One policy definition can therefore enforce in CI and at admission. Mutation in Gatekeeper is a separate feature you opt into. Kyverno is a policy engine that does both validation and mutation in declarative YAML (a plain-text configuration format) instead of Rego. The API server calls these webhooks over TLS (Transport Layer Security, the encryption behind HTTPS) on the hot path of every write, so they sit inside your latency and availability budget. Hold that thought until we reach failure modes. Kubernetes also ships a built-in validating layer called Pod Security Admission. Use it for the baseline Pod Security Standards, the stock profiles that cover the obvious dangers, and reach for Gatekeeper or Kyverno when a control goes past what those profiles can say: custom labels, allowed image registries, logic that compares one field against another.

Step 1: install Gatekeeper

The install gives you two workloads. A controller answers the webhook calls the API server makes, and an audit Pod re-scans what is already running in the cluster. Get both healthy before you write a single line of policy.

helm repo add gatekeeper https://open-policy-agent.github.io/gatekeeper/charts
helm install gatekeeper gatekeeper/gatekeeper \
  --namespace gatekeeper-system --create-namespace --version 3.18.0

NAME: gatekeeper
NAMESPACE: gatekeeper-system
STATUS: deployed
REVISION: 1
 
$ kubectl get pods -n gatekeeper-system
NAME                                             READY   STATUS    RESTARTS   AGE
gatekeeper-audit-6d9c8b7f4-x2k9p                 1/1     Running   0          72s
gatekeeper-controller-manager-7b5c9d6f8-ab12c    1/1     Running   0          72s
gatekeeper-controller-manager-7b5c9d6f8-de34f    1/1     Running   0          72s
gatekeeper-controller-manager-7b5c9d6f8-gh56k    1/1     Running   0          72s

Step 2: a ConstraintTemplate and a Constraint

Gatekeeper enforcement takes two objects, and you need both. A ConstraintTemplate is the recipe card. It holds the Rego that decides what counts as a violation, and it registers a brand-new resource kind with the cluster, here K8sDenyPrivileged. A recipe card on the shelf feeds nobody. A Constraint is one instance of that new kind: it says which resource kinds and which namespaces the rule applies to, and it sets the enforcementAction. The template writes the rule. The constraint switches it on for specific resources. That split pays for itself in reuse, because one template can back many constraints. The same privileged-container rule can run with deny in a namespace that is in PCI DSS scope and with dryrun everywhere else, and a shared template library carries one control definition across every cluster you operate. Apply a Constraint whose template was never installed and the API server turns it down flat. K8sDenyPrivileged is not a registered CRD (custom resource definition, the way you teach Kubernetes a new object type), so there is nothing for it to create.

apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: k8sdenyprivileged
spec:
  crd:
    spec:
      names:
        kind: K8sDenyPrivileged      # the new CRD kind the Constraint instantiates
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8sdenyprivileged
 
        violation[{"msg": msg}] {
          c := input.review.object.spec.containers[_]
          c.securityContext.privileged == true
          msg := sprintf("Privileged container not allowed: %v", [c.name])
        }

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sDenyPrivileged            # must match template's crd.spec.names.kind
metadata:
  name: no-privileged
spec:
  enforcementAction: deny          # use dryrun first in a live cluster (Step 4)
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]

kubectl apply -f template.yaml
kubectl apply -f constraint.yaml

constrainttemplate.templates.gatekeeper.sh/k8sdenyprivileged created
k8sdenyprivileged.constraints.gatekeeper.sh/no-privileged created

Step 3: watch it reject a real Pod

Now break the control the way a rushed engineer actually breaks it. A direct kubectl apply that never went near CI.

apiVersion: v1
kind: Pod
metadata:
  name: rogue
spec:
  containers:
    - name: app
      image: nginx:1.27
      securityContext:
        privileged: true

kubectl apply -f rogue-pod.yaml
echo "exit code: $?"

Error from server (Forbidden): error when creating "rogue-pod.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [no-privileged] Privileged container not allowed: app
exit code: 1

The request never reached etcd. The webhook answered Forbidden, the message names the constraint that fired ([no-privileged]) and repeats the exact reason your Rego produced, and kubectl exits non-zero. That exit code is what lets a wrapper script or a GitOps controller, the agent that keeps your cluster matching Git, treat the rejection as a hard failure rather than a warning nobody reads. The identical Pod is blocked whether it arrives from a pipeline, from a Helm release, or from a tired human at a terminal. That is the control the auditor asked you to prove.

Step 4: roll out with dryrun, not deny

Banks do not switch on a new fraud rule and start declining cards the same morning. They run it in shadow for a week and read what it would have blocked. Do the same here. Never flip a new control straight to deny in a live cluster, because you will block legitimate workloads you forgot existed and page yourself at 2 a.m. Set enforcementAction: dryrun first. In dryrun the webhook admits everything, while the audit Pod records every violation in the Constraint's status, so you can measure blast radius against real traffic before anything is blocked. The audit Pod re-evaluates on an interval, 60 seconds by default, so a freshly applied constraint's status can lag a minute before totalViolations is worth trusting. Measure, do not guess. Read the count, read the offending resources, fix or exempt the ones that are legitimate, then change that one field to deny, namespace by namespace. Hand an auditor the dryrun violation report, the remediation you did, and the date you switched to enforce, and you have handed them a clean piece of change evidence.

kubectl get k8sdenyprivileged no-privileged \
  -o jsonpath='{.status.totalViolations}{"\n"}'
kubectl get k8sdenyprivileged no-privileged -o json \
  | jq -r '.status.violations[] | "\(.namespace)/\(.name): \(.message)"'

2
payments/legacy-agent: Privileged container not allowed: agent
default/debug-shell: Privileged container not allowed: shell

Kyverno: validate and mutate without Rego

Rego is a language, and every language has a learning curve. Teams that would rather not climb it reach for Kyverno, where a policy is plain YAML and one engine covers both jobs. A validate rule with failureAction: Enforce rejects a privileged Pod exactly the way Gatekeeper does. A mutate rule does the thing a validating webhook cannot do at all: it edits the object on the way in. The rule below stamps a default data-classification label onto any Pod that shows up without one, so a required-label control heals itself instead of becoming a wall of rejections. Validating admission answers yes or no. Mutating admission changes the object until the answer is yes. Kyverno also runs background scans, which means it reports violations on Pods that were already running long before the policy landed. Most production clusters run both styles of rule side by side: mutate to set safe defaults, validate to hold the line on the things you will not bend on.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: pod-standards
spec:
  background: true
  rules:
    # VALIDATE: reject privileged Pods (same control as the Gatekeeper example)
    - name: disallow-privileged
      match:
        any:
          - resources:
              kinds: ["Pod"]
      validate:
        failureAction: Enforce
        message: "Privileged containers are not allowed."
        pattern:
          spec:
            containers:
              - =(securityContext):
                  =(privileged): "false"
    # MUTATE: add a default data-classification label if one is missing
    - name: default-data-classification
      match:
        any:
          - resources:
              kinds: ["Pod"]
      mutate:
        patchStrategicMerge:
          metadata:
            labels:
              +(data-classification): "unclassified"

There is a by-product here worth naming. Every decision you have watched, each admit, each reject, each dryrun violation, is logged with a timestamp, the resource, and the constraint that fired. That stream is the operating-effectiveness evidence a SOC 2 Type II auditor is after, because it shows the control ran against every resource the cluster admitted rather than against a handful you picked yourself. Be precise about what it proves, though. A passing admission check is evidence toward CIS control 5.2.2, not a certificate that you meet it. The auditor still wants your written narrative and still chooses their own sampling method. The next lesson, Evidence automation, turns that admission log into proof you never have to assemble by hand.

Try this

Work through “Kyverno: validate and mutate without Rego” yourself on a sandbox you can throw away, following the commands above in order. Then break one step deliberately and re-run, so you have seen the failure before it finds you.

Takeaway

The trap worth remembering here: a dead webhook or a wide namespace exclusion fails open, quietly. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related