SVIDs & rotation
X.509 for mTLS, JWT for requests, short lifetimes.
A hotel hands you two things at check-in. A room key that opens your door and keeps working for the whole stay. A signed breakfast voucher, good for one meal at one restaurant. Same guest, same front desk, two shapes of permission, and nobody tries to open a door with the voucher.
SVIDs split the same way. SVID stands for SPIFFE Verifiable Identity Document, the credential a workload actually holds. SPIFFE (Secure Production Identity Framework For Everyone) is the open standard that says what one looks like, and SPIRE (the SPIFFE Runtime Environment) is the software that hands them out. The X.509 form is a short-lived TLS certificate (Transport Layer Security, the encryption behind the padlock in your browser) and it secures a connection. The JWT form (JSON Web Token, a small signed bundle of claims) authenticates one request. Both carry the same SPIFFE ID. Both get checked against the same trust bundle. Both expire fast enough that a stolen one is worth very little. What follows is how to pull each form off a running host, read what it really claims, and watch the runtime swap it out underneath a live process.
What an X.509-SVID Actually Claims
An ordinary web certificate is a laminated card that says "I am api.example.com", signed by an authority your browser already trusts. An X.509-SVID is the same laminated card with a different name printed on it. Instead of a hostname, it carries the workload's SPIFFE ID in the URI SAN (Subject Alternative Name, the certificate field that lists which names a certificate is good for; the URI entry holds a name in web-address shape). The old-fashioned place to write down who a certificate belongs to, the Subject and its Common Name, is not used for identity here at all. SPIFFE requires verifiers to decide on the URI SAN and forbids falling back to the Subject, and SPIRE does not even bother putting a Common Name in the certificate. That is why you can hand the same certificate to services in three different clusters and they all agree on who you are.
Verification runs against the trust bundle, the set of public CA (certificate authority) roots for a trust domain. When two workloads open an mTLS connection (mutual TLS, where both ends present a certificate rather than only the server), a successful handshake proves exactly three things. The peer holds the private key matching the certificate it presented. That certificate chains to a root you already trust. The URI SAN inside it reads spiffe://acme.internal/ns/prod/sa/frontend. No password moved across the wire. No API key sat in an environment variable waiting to be scraped out of a crash dump.
Be precise about the rest, because this is where teams overtrust the handshake. mTLS says nothing about authorization: a valid identity is a name, not a permission, and every workload in the trust domain can present one. It proves who is on the other end of this network connection, not who started the request. If traffic flows client, then ingress gateway, then backend, the backend's peer is the gateway. Envoy (the proxy that sits beside each workload in a service mesh) will forward the original identity in the x-forwarded-client-cert header, but that is a claim the gateway makes, believed because you trust the gateway, not because anything is cryptographically bound to the request. And on a pooled HTTP/2 connection, where thousands of requests are multiplexed over one long-lived socket, they all share a single handshake, so per-request identity is not something the certificate can give you. That gap is the hole JWT-SVIDs exist to fill.
Pull One Off the Wire
The workload never generates or stores a key. It asks the front desk. The front desk is the Workload API, a gRPC service (gRPC being a fast binary way for one program to call functions in another) that the SPIRE Agent exposes on a local unix domain socket, which is a file on disk that behaves like a network socket and is reachable only from that host. There is no token to present. The socket plus the kernel does the authentication: the agent asks the kernel who is on the other end of the connection, matches those process facts against registration entries by selector, then hands back a freshly minted private key, the signed certificate, and the current trust bundle. Libraries find the socket through the SPIFFE_ENDPOINT_SOCKET environment variable, which holds a URI such as unix:///run/spire/agent/public/api.sock rather than a bare path. The command line takes a plain path via -socketPath.
# Do by hand what a mesh sidecar or a SPIFFE library does on every startup.spire-agent api fetch x509 \-socketPath /run/spire/agent/public/api.sock \-write /run/spire/svids
Received 1 svid after 4.148031msSPIFFE ID: spiffe://acme.internal/ns/prod/sa/frontendSVID Valid After: 2026-07-14 09:00:12 +0000 UTCSVID Valid Until: 2026-07-14 10:00:12 +0000 UTCCA #1 Valid After: 2026-07-14 08:00:00 +0000 UTCCA #1 Valid Until: 2026-07-15 08:00:00 +0000 UTCWriting SVID #0 to file /run/spire/svids/svid.0.pem.Writing key #0 to file /run/spire/svids/svid.0.key.Writing bundle #0 to file /run/spire/svids/bundle.0.pem.
One hour of certificate, sitting under a CA root good for a day. Those are SPIRE's defaults, and the relationship between those two numbers matters later. First, read the certificate itself instead of trusting the CLI's summary of it.
# Who does the certificate say you are? Not the subject. The URI SAN.openssl x509 -in /run/spire/svids/svid.0.pem -noout -subject -issueropenssl x509 -in /run/spire/svids/svid.0.pem -noout -ext subjectAltNameopenssl x509 -in /run/spire/svids/svid.0.pem -noout -datesopenssl verify -CAfile /run/spire/svids/bundle.0.pem /run/spire/svids/svid.0.pem
subject=C = US, O = SPIREissuer=C = US, O = SPIFFEX509v3 Subject Alternative Name:URI:spiffe://acme.internal/ns/prod/sa/frontendnotBefore=Jul 14 09:00:12 2026 GMTnotAfter=Jul 14 10:00:12 2026 GMT/run/spire/svids/svid.0.pem: OK
Read that subject line again. It says O = SPIRE and nothing else. No namespace, no service account, no team, no cluster, and no Common Name to be tempted by. That is deliberate. SPIFFE moved identity into one machine-readable field so nobody ever has to parse a Common Name with a regular expression again. Notice too what openssl verify did and did not do. OK means the chain is intact and the signature checks out. It says nothing about which SPIFFE ID is inside. A verifier that stops at chain validation will cheerfully accept spiffe://acme.internal/ns/dev/sa/scratch-job from anyone in the trust domain, because that certificate is also perfectly valid. Chain, then URI SAN, then authorization. All three, every time.
When the Connection Cannot Carry the Identity
Some errands cannot be done with the room key. Mail routed through a sorting office arrives in the sorting office's van, and the recipient sees the van. Any hop that terminates TLS and opens a fresh connection does that to your certificate identity: the original one stops at the door. A JWT-SVID is the signed slip you put inside the envelope so it survives the sorting office. It carries the workload's SPIFFE ID as the sub (subject) claim, plus an aud (audience) claim naming who is allowed to accept it.
# 'aud' pins the token to ONE recipient. Ask for the audience you are calling.spire-agent api fetch jwt \-audience api.prod.svc \-socketPath /run/spire/agent/public/api.sock
token(spiffe://acme.internal/ns/prod/sa/frontend):eyJhbGciOiJFUzI1NiIsImtpZCI6IkxxWU1JN0k2bnJUNGtGUHZQdlhNRmNDUFh0V3ZVSmowIiwidHlwIjoiSldUIn0.eyJzdWIiOiJ...bundle(spiffe://acme.internal):{"keys": [{"kty": "EC","kid": "LqYMI7I6nrT4kFPvPvXMFcCPXtWvUJj0","crv": "P-256","x": "iRHFYAqz4Bl0dgDdvS3EIRXAcYlq7q9SkGx0YHhCwXk","y": "vLGjb2r2SbWJ9nEvhJ3sMZ2mNAcMOa6r8yF7hE0dLpQ"}]}
The command prints two things because a receiver needs two things: the token, and the public keys that prove it was signed by this trust domain. Decode the middle segment of the token to see what the receiver will actually be checking.
TOKEN=$(spire-agent api fetch jwt -audience api.prod.svc \-socketPath /run/spire/agent/public/api.sock | sed -n '2p' | tr -d '[:space:]')# JWTs use base64url, so pad it properly rather than piping to plain base64 -d.python3 -c 'import base64, json, sysp = sys.argv[1].split(".")[1]print(json.dumps(json.loads(base64.urlsafe_b64decode(p + "=" * (-len(p) % 4))), indent=2))' "$TOKEN"
{"sub": "spiffe://acme.internal/ns/prod/sa/frontend","aud": ["api.prod.svc"],"exp": 1784019912,"iat": 1784019612}
Those two numbers are plain Unix timestamps: iat is when the token was issued, exp is when it dies. Subtract them and you get 300 seconds. Five minutes, SPIRE's default for JWT-SVIDs, and much shorter than the certificate's hour for a reason we get to in a moment. A receiver has four jobs here, and skipping any one of them turns the token into decoration. Verify the signature with the public key from the trust bundle whose kid (key ID, the label that says which key signed this) matches the token header. Reject anything whose header says alg: none, or names an algorithm you were not expecting. Require that aud contains your own name, so a token minted for api.prod.svc is dead weight at billing.prod.svc. Require that sub is a SPIFFE ID you meant to allow. One more detail catches people out: the bundle often carries two JWT public keys at once, because SPIRE keeps the outgoing key published until the last token it signed has expired. A receiver that caches the key set once at startup starts rejecting perfectly valid tokens the day the signing key turns over.
Rotation Is the Revocation Model
A keycard that stops working at checkout does not need a lost-and-found process. SVIDs are built on the same idea, and this is the part people file under "annoying" when it is actually the whole security model. The SPIRE Agent does not answer one question and hang up. It holds a streaming connection open per workload, and when a certificate reaches half its life the agent fetches a replacement and pushes it down that same stream. On a one-hour SVID you get a new one every thirty minutes, with thirty minutes of overlap in which to install it. You can watch it happen.
# 'watch' keeps the Workload API stream open and prints every update it receives.spire-agent api watch -socketPath /run/spire/agent/public/api.sock
Received 1 svid after 1.596889msSPIFFE ID: spiffe://acme.internal/ns/prod/sa/frontendSVID Valid After: 2026-07-14 09:00:12 +0000 UTCSVID Valid Until: 2026-07-14 10:00:12 +0000 UTCCA #1 Valid After: 2026-07-14 08:00:00 +0000 UTCCA #1 Valid Until: 2026-07-15 08:00:00 +0000 UTCReceived 1 svid after 30m0.314179sSPIFFE ID: spiffe://acme.internal/ns/prod/sa/frontendSVID Valid After: 2026-07-14 09:30:12 +0000 UTCSVID Valid Until: 2026-07-14 10:30:12 +0000 UTCCA #1 Valid After: 2026-07-14 08:00:00 +0000 UTCCA #1 Valid Until: 2026-07-15 08:00:00 +0000 UTC
Nothing was revoked. Nothing needed to be. SPIFFE deployments almost never ship a CRL (certificate revocation list, the traditional "these certificates are cancelled" file) because expiry does that job without a distribution problem to solve. Now the honest limit. If a workload is compromised at 09:05, deleting its registration entry with spire-server entry delete -entryID <id> stops the next issuance and the agent stops handing the identity out at its next sync, but the certificate already in the attacker's hands keeps working until 10:00. Short lifetimes bound that window, they do not close it. When you need it closed now, SPIRE 1.9 added a real lever that every release since has carried: spire-server localauthority x509 taint -authorityID <id> marks a signing authority as compromised, which forces every agent to rotate the SVIDs that authority signed, and revoke then drops it from the trust bundle so anything still holding an old certificate gets refused at the handshake.
-write is fine for inspection and wrong as a production pattern unless something keeps re-reading the files. An application that loads svid.0.pem at startup and caches the parsed certificate keeps presenting it long after the agent has delivered a replacement, and at the expiry minute every new handshake fails with x509: certificate has expired or is not yet valid. Established connections often survive, so the failure looks partial and random rather than obvious, and it lands roughly one certificate lifetime after deploy, which is usually the middle of the night. Use a SPIFFE library that holds the stream open, or let the mesh sidecar own the certificate. Never pin one SVID for the life of a process.Setting Lifetimes Without Breaking Things
Two dials control lifetime. Fleet-wide defaults live in the SPIRE Server config, and any single registration entry can override them for one identity.
server {bind_address = "0.0.0.0"bind_port = "8081"trust_domain = "acme.internal"data_dir = "/opt/spire/data/server"log_level = "INFO"# How long the signing CA itself lives.ca_ttl = "24h"# Handed to every workload unless its own entry overrides them.default_x509_svid_ttl = "1h"default_jwt_svid_ttl = "5m"}# plugins { ... } omitted: datastore, node attestor, key manager
The parent ID below is the identity of the agent that will serve this workload. On Kubernetes that ID is built by the PSAT node attestor (Projected Service Account Token, a short-lived token the kubelet mints for a pod so the node can prove which cluster it belongs to), and it ends in the node's own UID. The TTL flags (time to live, how long the issued credential stays good) take a plain number of seconds.
# The TTL flags take SECONDS. 15 minutes of certificate, 2 minutes of token.spire-server entry create \-parentID spiffe://acme.internal/spire/agent/k8s_psat/prod-cluster/9c2e1d7b-40f5-4a11-8f2e-6d3b0c7a1e94 \-spiffeID spiffe://acme.internal/ns/prod/sa/payments \-selector k8s:ns:prod \-selector k8s:sa:payments \-x509SVIDTTL 900 \-jwtSVIDTTL 120
Entry ID : 4b1f3a2c-7e55-4b64-9a4f-1a2c9e6d0f31SPIFFE ID : spiffe://acme.internal/ns/prod/sa/paymentsParent ID : spiffe://acme.internal/spire/agent/k8s_psat/prod-cluster/9c2e1d7b-40f5-4a11-8f2e-6d3b0c7a1e94Revision : 0X509-SVID TTL : 900JWT-SVID TTL : 120Selector : k8s:ns:prodSelector : k8s:sa:payments
The entry wins over the server default. Those flags want seconds, not a duration string, which bites everyone once: -x509SVIDTTL 15 buys you a fifteen second certificate, not fifteen minutes. Tighten the identities that touch money, keys, or customer data, leave the rest on the default, and confirm the result with spire-server entry show -spiffeID spiffe://acme.internal/ns/prod/sa/payments instead of assuming the flag took.
Here is the trap that quietly undoes your tuning. SPIRE will not sign a certificate that outlives the authority signing it. Ask for a longer life than the current signing key has left, and the lifetime is truncated to that key's own expiry, with no error anywhere near the workload.
# server.conf now says default_x509_svid_ttl = "12h". ca_ttl is still "24h".spire-agent api fetch x509 -socketPath /run/spire/agent/public/api.sock
Received 1 svid after 4.021884msSPIFFE ID: spiffe://acme.internal/ns/prod/sa/frontendSVID Valid After: 2026-07-14 09:00:12 +0000 UTCSVID Valid Until: 2026-07-14 16:41:03 +0000 UTCCA #1 Valid After: 2026-07-13 16:41:03 +0000 UTCCA #1 Valid Until: 2026-07-14 16:41:03 +0000 UTC
You asked for twelve hours and got seven hours and forty minutes, and the tell is sitting right there in the output: the SVID's Valid Until is the same timestamp as the CA's, to the second. Watch it again tomorrow and you will get a different, equally arbitrary number, because it tracks whatever the signing key has left. SPIRE flags this arrangement at startup, warning that the configured default_x509_svid_ttl is too high for the configured ca_ttl and naming both numbers that would fix it (with a 24 hour CA it points you at a 4 hour SVID TTL, or a 72 hour CA). The working rule is ca_ttl at least six times your longest SVID lifetime. When certificates come back shorter than you configured, compare those two Valid Until lines before you go hunting anywhere else.
Shorter lifetimes shrink the theft window and grow the bill. Every rotation is a signing operation on the server, and since agents renew at half-life, the steady rate is roughly the number of workloads divided by half the TTL. Twenty thousand workloads on a one-hour certificate is about eleven signatures a second. Move that same fleet to five-minute certificates and it is around one hundred and thirty a second, on a component whose outage means nothing new can start and nothing existing can renew. The other end of the dial hurts differently. With very short lifetimes, clock skew stops being cosmetic. Certificate verification has no built-in grace period, so a receiver whose clock runs two minutes fast rejects a brand new certificate as not yet valid, and on a five-minute credential that is nearly half its life gone. Run NTP (Network Time Protocol, the service that keeps machine clocks agreed) and treat skew alerts as security alerts. The stock hour and five minutes are good numbers. Change them with evidence, and never lengthen them to cut rotation churn, because that rebuilds the long-lived credential problem SPIFFE exists to delete.
In a Mesh, You Watch Instead of Fetch
Running fetch loops and reload logic inside every service does not scale, and in a mesh you do not have to. Istio's per-pod agent gets the workload certificate, keeps it in memory rather than on disk, and serves it to the Envoy sidecar over SDS (Secret Discovery Service, the gRPC channel Envoy uses to receive certificates and keys). Envoy swaps in the new certificate without dropping established connections. The default workload certificate lives 24 hours and the agent renews at half of that, set by the istio-agent environment variables SECRET_TTL and SECRET_GRACE_PERIOD_RATIO, which default to 24h and 0.5. You can read the live window from outside, without touching the workload.
istioctl proxy-config secret deploy/frontend -n prod
RESOURCE NAME TYPE STATUS VALID CERT SERIAL NUMBER NOT AFTER NOT BEFOREdefault Cert Chain ACTIVE true 274054158399344857396100907785838425513 2026-07-15T09:00:12Z 2026-07-14T09:00:12ZROOTCA CA ACTIVE true 198443259901137842277095901633120148071 2036-07-10T12:00:00Z 2026-07-13T12:00:00Z
default is this pod's X.509-SVID, one day long. ROOTCA is the trust bundle root used to verify peers, and it is deliberately long-lived (ten years on Istio's self-signed default). The same identity rules apply inside the mesh as outside it, which you can prove by pulling the certificate out of Envoy's config dump and reading the field that matters.
istioctl proxy-config secret deploy/frontend -n prod -o json \| jq -r '.dynamicActiveSecrets[] | select(.name=="default")| .secret.tlsCertificate.certificateChain.inlineBytes' \| base64 --decode \| openssl x509 -noout -subject -ext subjectAltName -dates
subject=X509v3 Subject Alternative Name: criticalURI:spiffe://cluster.local/ns/prod/sa/frontendnotBefore=Jul 14 09:00:12 2026 GMTnotAfter=Jul 15 09:00:12 2026 GMT
An empty subject, and the SPIFFE ID in a URI SAN marked critical so no verifier is allowed to skip it. Istio issues its own SVIDs from its own CA by default, under the cluster.local trust domain, but the shape of the credential is the one you have been reading all lesson.
That name still is not a permission. Turning it into one takes two resources, both on the current security.istio.io/v1 API.
apiVersion: security.istio.io/v1kind: PeerAuthenticationmetadata:name: defaultnamespace: prodspec:mtls:mode: STRICT---apiVersion: security.istio.io/v1kind: AuthorizationPolicymetadata:name: payments-allow-checkoutnamespace: prodspec:selector:matchLabels:app: paymentsaction: ALLOWrules:- from:- source:principals: ["cluster.local/ns/prod/sa/checkout"]to:- operation:methods: ["POST"]paths: ["/v1/charges"]
STRICT mode makes the sidecar refuse plaintext, so there is always a peer identity to check. principals is that peer identity written the way Istio writes it, trust domain first and the spiffe:// scheme dropped, so spiffe://cluster.local/ns/prod/sa/checkout becomes cluster.local/ns/prod/sa/checkout. Get that string wrong and the policy silently matches nothing. One behaviour surprises people: as soon as any ALLOW policy selects a workload, everything that fails to match a rule on that workload is denied, so the first policy you apply to payments is also the moment every unlisted caller starts getting 403s. If you need per-request identity instead of per-connection, the sibling field requestPrincipals matches a verified token in issuer/subject form, and it only means anything once a RequestAuthentication resource has actually verified that token.
The secret output is also your proof that rotation is real. Run istioctl proxy-config secret today, run it again after one full certificate lifetime, and compare the serial number and NOT AFTER. If both moved and nobody paged you, the whole chain works: the agent renewed, Envoy reloaded, no connection noticed. If the serial number is identical a day later, nothing is rotating, and what you actually have is a long-lived credential wearing a short-lived costume.
Try this
Run openssl x509 -in /run/spire/svids/svid.0.pem -noout -subject -issuer on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.
Takeaway
The trap worth remembering here: jWT-SVIDs are bearer tokens, so treat them like cash. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.