Vault Agent and secret injection

Templates, renewals, and keeping apps dumb.

In short. The best Vault integration is often one the application never notices.

The best Vault integration is often one the application never notices. Vault Agent (and related injectors/operators) authenticates with the workload identity, renders secrets to files or environment sinks, and renews leases before expiry. The app reads a file on disk the way it always did — except the file's contents rotate underneath it.

Pushing the Vault SDK into every microservice creates version drift, inconsistent error handling, and secret-zero mistakes. Agent centralizes that complexity. On Kubernetes, the Agent Injector or Secrets Store CSI / Vault Secrets Operator are the usual delivery belts; on VMs, Agent as a systemd service is enough.

Agent auto-auth and templates

Configure an auto_auth method (Kubernetes, AppRole, AWS, GCP, Azure, etc.) and sink a token. Templates use Consul Template-compatible syntax to write files atomically. A command can SIGHUP nginx or bounce a unit after render. Set file permissions to 0640 owned by the app user — world-readable secret files recreate the old problem.

auto_auth {
  method "kubernetes" {
    config = {
      role = "webapp"
    }
  }
  sink "file" {
    config = {
      path = "/vault/token"
    }
  }
}
 
template {
  source      = "/etc/vault/templates/app.env.ctmpl"
  destination = "/app/config/app.env"
  perms       = "0640"
  command     = "systemctl reload myapp"
}

# render once in lab
vault agent -config=/etc/vault.d/agent.hcl -exit-after-auth=false
# or check a rendered file:
ls -l /app/config/app.env
grep -c PASSWORD /app/config/app.env

-rw-r----- 1 vault app 128 Jul 24 12:00 /app/config/app.env
1

Kubernetes delivery choices

Injector annotations mutate pods with a sidecar. CSI mounts secrets as volumes. External Secrets / VSO sync into Kubernetes Secret objects (useful, but then etcd holds the value again — combine with encryption at rest and tight RBAC). Pick one primary pattern per platform team to avoid three partial integrations.

Going deeper

Atomic writes matter: Agent should write to a temp file and rename into place so apps never read half-rendered secrets. Set error_on_missing_key appropriately — failing closed beats silently writing empty passwords on a Vault blip.

On Kubernetes, decide whether the app may see a Kubernetes Secret object at all. Injector sidecars and CSI can keep values off the API. If you sync into Secrets for convenience, enable encryption at rest and lock RBAC so get secrets is not world-readable in the namespace.

Load-test renewals. A thousand Agents renewing on the same minute create spikes. Stagger TTLs and cache where safe. Watch Agent logs for permission denied after policy changes — apps will otherwise fail later when a lease finally expires.

Try this

In a lab, run Agent with a static template that reads a KV path and writes a file; update the KV secret and confirm the file changes after renew/render.

vault kv put secret/agentdemo password=first
# start agent with template… then:
vault kv put secret/agentdemo password=second
sleep 5
cat /app/config/app.env

PASSWORD=second
# SUCCESS — file reflects new version without app speaking Vault

Takeaway

Vault Agent keeps apps dumb and renewals correct. Authenticate with platform identity, render narrowly, reload deliberately, and remember the file is still a secret.

Next: turn on audit devices so every read has a forensic trail.

Related