Governance: Sentinel, control groups & namespaces

Policy-as-code guardrails, four-eyes approval, and hard multi-tenancy.

At 2 a.m., from a laptop on a home wifi connection, a developer reads every secret under secret/data/prod/*. Every check passes. Nothing is broken. An ACL (access-control list, the rule that says which identity may touch which path) only ever answers one question: is this identity allowed on this path? Governance asks the questions that actually keep you up at night. Should this be allowed right now? Does it need a second person to sign off? Are the tenants really separated, or do they only look separated? Three things answer those: conditional guardrails layered on top of ACLs, multi-party approval, and hard multi-tenancy. Every mature secrets platform ends up with all three.

Guardrails that read the room

Your office badge opens the server-room door. A guard standing beside that door can still stop you, because the guard sees what the badge reader cannot: the hour, your face, the box under your arm. An ACL is the badge reader, and it knows one thing, this path plus this capability, yes or no. A guardrail policy is the guard. Vault Enterprise writes those guards in a language called Sentinel; other systems do the same job with OPA (Open Policy Agent, an open-source policy engine). Either can enforce rules an ACL has no way to express. Generate a root token only from the corporate CIDR (Classless Inter-Domain Routing, the shorthand for an address range such as 10.0.0.0/8). No secret reads outside business hours for this role. MFA (multi-factor authentication, the second proof beyond a password) required on this mount. Vault splits them in two: role-governing policies run on every request from an identity carrying that role, and endpoint-governing policies attach to specific paths.

Authorization stops being a fixed answer and starts depending on circumstance. Same person, same path, different verdict, because the hour, the source address or the login method changed. Go back to the developer at 2 a.m. Their ACL grants read on secret/data/prod/*, so the badge reader opens. The Sentinel rule sees a home IP address and a session with no MFA step behind it, and refuses. That is precisely the control you want sitting in front of production.

import "sockaddr"
import "mfa"
 
# only from the corporate CIDR
cidr = rule { sockaddr.is_within(request.connection.remote_addr, "10.0.0.0/8") }
 
# and only with a validated MFA method
mfa_ok = rule { mfa.methods.duo.valid }
 
main = rule when request.path matches "secret/data/prod/.*" {
  cidr and mfa_ok
}

Control groups: two people or nobody

A bank vault with two locks and two keyholders is not an insult to either person. It removes the possibility that one person acting alone can open it. Control groups do that for a secret. The requester asks for the value and the request stops there, pending, released to nobody. One or more named approvers sign off. Only then can the requester unwrap the value and read it. That is what people mean by four-eyes: two pairs of eyes on one action. Put it on the credentials where a single bad night ends badly, such as break-glass logins, production database root passwords, and the keys that decrypt customer data.

The log is the other half of the deal. It records who asked, who approved and at what time, so a sensitive read becomes an event somebody can review later instead of a line nobody ever opens. Start with the paths that hurt most: anything equivalent to production database root, your signing keys, cross-tenant admin roles. Sprinkling control groups over low-value paths teaches everyone that approval requests are noise, and then they stop reading them.

vault write sys/config/control-group \
  max_authorizations=2
 
vault write sys/control-groups/authorize \
  accessor=CG_ACCESSOR
# approver runs authorize; requester then reads the secret

Success! Data written to: sys/config/control-group
 
Success! Control group authorized (1 of 2)
...
Success! Control group authorized (2 of 2) — secret released

Namespaces: real walls between tenants

Templated policy is an open-plan office with a rule that says do not read the papers on someone else's desk. It holds until one rule is written wrong. A namespace is a separate floor with its own front door. Each namespace carries its own policies, its own auth methods, its own secret engines and its own identities, handed to a tenant team that runs all of it while seeing nothing that belongs to anyone else. Soft multi-tenancy sits one policy typo away from a leak. Hard multi-tenancy is a structural boundary, and a typo cannot cross it.

Reach for namespaces when the tenants are genuine trust boundaries: separate business units, separate customers, separate regulatory scopes. Then delegate the administration, or your central platform team turns into the queue everyone waits in. A tenant admin can enable a KV (key-value) engine and Kubernetes auth inside their own namespace on a Tuesday afternoon without filing a ticket, and still see nothing of the sibling namespaces or the root.

vault namespace create tenant-a
vault namespace create tenant-b
vault namespace use tenant-a
vault secrets enable -path=kv kv-v2
vault auth enable kubernetes

Success! Created namespace tenant-a!
Success! Created namespace tenant-b!
Success! Enabled the kv-v2 secrets engine
Success! Enabled kubernetes auth method
# tenant-a admins cannot see root or tenant-b mounts

Put the rules in Git and try to break them

Sentinel rules, control groups and namespace boundaries are real only when they live somewhere reviewable and somebody is watching them. Keep the guardrail policies in Git, behind the same pull-request review your application code gets. Raise an alarm when a control group is bypassed, or when a namespace admin quietly grants themselves a new engine. Then test the rules, which is the step almost everyone skips.

A policy with a typo that silently evaluates to allow is worse than no policy, because the whole team believes it is covered. So write the negative tests. Assert that a read of a production secret from 8.8.8.8 at midnight fails, even when the ACL on its own would have said yes. If you have never watched a guardrail deny something on purpose, you do not know it works.

When to add each layer

Begin with ACLs and templated paths, and stay there for as long as the path is the only thing that matters. Add Sentinel or OPA the day context starts to matter: time of day, source network, whether MFA actually happened. Add control groups when a regulation or your own internal policy demands two signatures on a specific path. Add namespaces when tenants must not share auth methods, audit streams or operators. Every layer costs someone's week, so add it when the risk earns the machinery, not because the feature exists in the product.

Write down which paths need control groups, and which namespace belongs to which business unit. No auditor accepts "we use Sentinel" as evidence. What counts is the policy file under version control, the review history attached to it, and the alert in your SIEM (security information and event management system, where logs go to be watched) that fires when somebody tries to route around it.

vault namespace use tenant-a
vault policy list
vault read sys/namespaces/tenant-a

default
app-read
tenant-admin
 
id          tenant-a/
path        tenant-a/
# isolated policy and auth namespace — structural tenant boundary

Stopping a stolen token from growing

Endpoint-governing policies attach to a path. Blocking delete on secret/data/prod/* for everyone except the break-glass roles is the classic example. Role-governing policies attach to an identity instead and run on every request that identity makes. Use both together. A stolen token rarely stops at reading one secret; it tries to enable a new auth method or attach a sudo policy to itself, and then it reads everything.

Namespace admins can run their own secret engines day to day without holding root on the parent cluster, which is the whole point of delegating. Keep watching anyway. Namespace creation and policy changes belong in the root audit stream where your team actually reads them. A tenant who grants themselves every power inside their own namespace still cannot touch a sibling namespace, and that containment is what you bought.

vault policy read sudo -format=json | jq ".data.policy" | head -5
vault read sys/internal/ui/mounts | jq "keys" | grep -c secret

"path \"*\" { capabilities = [\"create\"...]}"
# review quarterly — sudo count should be minimal
42

Approvers have to be a different set of people from requesters. When one person sits in both groups, four-eyes is a label stuck on a one-person process. Drive approver membership from your IdP (identity provider, the system holding your company's user accounts and groups) exactly as you drive engineering access, because a hand-kept approver list inside Vault goes stale within a quarter. People change teams. The list never notices.

Spell out in the delegation agreement which engines a tenant may enable. Leave it open and one tenant admin turns on a PKI (public key infrastructure) mount, and now the company runs an internal certificate authority nobody planned for, issuing certificates nobody tracks. Root operators keep a veto by using namespace policy templates that cap the dangerous mount types.

Governance also fails in a way no dashboard shows. Engineers decide the rules block real work, and they route around them in a chat thread. "Can you grab that value for me?" quietly becomes the production policy. Publish the escape hatch instead: a control group plus an audited break-glass path gives people a legitimate way through that leaves a record behind. Sentinel protects only what people cannot informally step around.

A regulatory mapping is a claim until you test it. Run the simulation. Attempt a production secret read from an untrusted address and confirm that Sentinel denies it while the ACL alone would have allowed it. Keep that output. It belongs in the compliance packet, next to the hash of the policy file that produced it.

Tag the Sentinel and control-group policy versions in Git so they line up with the Vault policy versions. During an incident the question is always the same, and always urgent: which version of the guardrail was live at 03:14, when that odd access succeeded? Without the tags you are reconstructing it from memory at the worst possible moment.

For production control groups, require two approvers from different manager chains. Two people in the same cost center approving each other passes the technical check and satisfies nobody who looks closely, neither the auditor nor the intent behind four-eyes.

Read the namespace admin actions in the root audit sink once a week. Delegated admin is still admin with a smaller blast radius, and scope creep shows up in the pattern of enable-mount calls long before any data leaves the building.

vault namespace use tenant-a
vault secrets list
vault policy read tenant-admin

Path    Type
----    ----
kv/     kv-v2
# tenant enabled only approved engines
 
path "kv/*" { capabilities = ["create","read","update","delete","list"] }

Governance is policy written as code, plus a change process wrapped around it. Path-based ACL policies do the daily work. Sentinel or OPA-style governance, an Enterprise feature in Vault, adds the sharper constraints for the cases where "this path is allowed" is too blunt an answer. Review the policy diffs the way you review application diffs: who asked for the access, what they needed it for, and when it expires.

Root and break-glass tokens deserve ceremony. Split the unseal or recovery keys across people who sit in different teams. Require dual approval for any policy that grants transit export or auth-method write. Audit every use, including the ones that looked routine. The quiet killer is a convenience alias somebody pasted into a runbook two years ago that happens to carry sudo, and which nobody has read since.

Try this

Pull the effective capabilities for a token, then point it at a path it has no business reaching. A guardrail you have never seen deny anything is a guess.

vault token capabilities secret/data/payments/db
vault token capabilities sys/policies/acl
vault policy read payments-read
vault kv get secret/payments/db 2>&1 | head -5

["read"]
["deny"]
 
path "secret/data/payments/*" {
  capabilities = ["read"]
}
 
Error reading secret/data/payments/db: Error making API request.
URL: GET .../secret/data/payments/db
Code: 403
# denied without payments-read — good

Takeaway

Run Vault the way you run any production control plane: small named policies, diffs a second person reviewed, short TTLs (time to live, how long a credential stays valid), and no casual use of root. Then put the context checks and the MFA requirement on the handful of paths where a path-only answer is not enough, because that is where the 2 a.m. read from a home address gets stopped.

Next: search your policies for anything carrying sudo or a wildcard write on secret/*. Replace each one with a named path set owned by a single team, and count how many turn out to be deletable outright.

Related