CoursesAdvanced secrets managementTransit: encryption & tokenization as a service

Transit: encryption & tokenization as a service

Data keys, convergent encryption, key rotation, and rewrap.

Advanced35 min · lesson 5 of 15

You need to encrypt a customer email column, and you would rather not keep the encryption key in your app's config file. Scrambling the text is the easy part. Storing the key, rotating it and deciding who is allowed to use it is where teams come unstuck. Vault's Transit engine does that job for you. Your app sends the readable value (plaintext), gets the scrambled value back (ciphertext), and never touches the key itself. The key stays inside Vault. Rotation and access control come with it, so you write no crypto plumbing of your own.

In plain terms
Transit works like the sealed-envelope counter at a bank. You slide a document through the slot and it comes back sealed in an envelope you cannot open. You never hold the master key; you only ever hold the sealed envelope. Lose the envelope and the contents are gone. Steal one and you still cannot seal new documents, because the key never left the counter.

Encrypt and decrypt without ever holding the key

Your app calls the encrypt endpoint with a key name, say orders, and gets back a ciphertext with the key version stamped on the front, like vault:v3:. To read the value later it calls decrypt on the same key. No key material ever lands in the application process. An attacker who owns that app can see whatever data is flowing through it right now, but cannot open the archive of old rows, and cannot walk off with a key that was never there.

Who may do what is set by Vault policy, and encrypt and decrypt are separate permissions. "This service may encrypt with the orders key but may not decrypt with it" is a rule Vault will actually enforce. Write-only encryption for the web frontend, decrypt reserved for the nightly batch job. That split is what lets you survive a compromised frontend without losing every encrypted record in the database along with it.

terminal
vault secrets enable transit
vault write -f transit/keys/orders
vault write transit/encrypt/orders plaintext=$(echo -n "4111 1111 1111 1111" | base64)
output
Success! Enabled the transit secrets engine
Key Version 1
ciphertext vault:v1:8SDd3W...
terminal
vault write transit/decrypt/orders ciphertext="vault:v1:8SDd3W..."
vault read transit/keys/orders
output
plaintext NDExMSAxMTExIDExMTEgMTExMQ== # base64 -> original card number
latest_version 1
min_decryption_version 0
# version metadata drives rotation policy

Rotation and rewrap, transparent by design

Every ciphertext carries the version of the key that produced it, and that one detail makes rotation cheap. Rotate the key and new writes come back as v2, while the old vault:v1: values still decrypt against the retained v1. When you want the old version gone for good, call the rewrap endpoint. It takes a vault:v1: ciphertext and hands back a vault:v2: one, doing the work inside Vault so the plaintext never comes out.

Then set min_decryption_version so Vault refuses anything older than v2 once the rewrap is done. Rotation stops being a project with a migration plan and a lost weekend. It turns into background hygiene: rewrap rows lazily as they are read, or push batches through during a quiet window.

Rotate without re-encrypting everything at once
1rotate key
new version v2, v1 retained
2new writes use v2
old vault:v1: still decrypts
3rewrap on read/batch
v1 ciphertext -> v2, no plaintext
4raise min version
v1 finally refused
The version prefix splits "rotate the key" from "re-encrypt the data", so you can do the second one at your own pace.
terminal
vault write -f transit/keys/orders/rotate
vault write transit/rewrap/orders ciphertext="vault:v1:8SDd3W..."
vault write transit/keys/orders/config min_decryption_version=2
output
latest_version 2
ciphertext vault:v2:9Klm4X... # rewrapped without plaintext exposure
min_decryption_version 2 # v1 ciphertext now rejected

Datakeys, convergent encryption, and tokenization

Pushing a 40 GB file through an API call is a bad plan. For bulk work Transit vends a datakey: a fresh symmetric key for that one object, returned twice. Once in the clear, so you encrypt locally at full speed and then throw it away. Once wrapped by the Transit key, so you store the wrapped copy next to the data. That pattern is envelope encryption, the same idea as a hotel where your card opens one room and the master stays behind the desk. You encrypt gigabytes locally while the key that protects all the little keys never leaves Vault.

Convergent encryption makes the same input produce the same ciphertext every time, so you can still index a column or spot duplicates without decrypting anything. The price is that whoever holds the ciphertext learns which rows are equal. Tokenization goes further. Transit swaps the sensitive value for a meaningless token and keeps the mapping inside Vault, so nothing downstream ever holds the real number and nobody can work backwards without calling Vault. That is the cheapest way to cut how much PII (personally identifiable information, things like names and card numbers) sits in your own tables.

Transit capabilities by service role
frontend / collector
encrypt only
write PII, never read back
tokenize
replace PAN with token
batch / analytics
decrypt scoped
specific keys, rate-limited
rewrap
migrate ciphertext versions
admin
rotate keys
scheduled hygiene
raise min version
retire old key material
Split the capabilities by policy path, so a compromised frontend never becomes a bulk-decrypt oracle.

Policy patterns: encrypt-only, sign, batch

Frontends get encrypt, and sign if they need it. Batch jobs get decrypt. Signing through Transit produces a signature while the private key stays put, which is handy for webhooks and audit chains where the receiver needs proof the message really came from you. Batch input endpoints let you send many values in one API call instead of one call per row. Each capability lives at its own policy path, so least privilege here comes down to listing the paths a token needs and stopping there.

Rate-limit the decrypt paths at the load balancer or in the service mesh (the layer that brokers traffic between your services). From Vault's side, a breach looks like a sudden run of decrypt calls. Alarm on decrypt volume per identity, and on decrypt traffic at 3am from a role that is only supposed to run in the nightly batch.

When Transit beats the KV store, and when it does not

Reach for Transit when your application has data of its own to protect and will store the result somewhere you control: a database column, a file on disk, a token in your own schema. Reach for the KV engine (key/value, where Vault holds the secret itself) when the thing you want back is the secret, like an API token or a password. Get the two backwards and you end up with ciphertext in KV and the key sitting in an environment variable, which gives you the weaknesses of both and the benefit of neither.

Transit keys cannot be exported, by design. If your disaster recovery plan says a human has to hold key material offline, that requirement belongs to an HSM-backed root (a hardware security module, a tamper-resistant box built to hold keys) plus a written break-glass procedure. It does not belong to a copy of the Transit key in a backup tarball.

Signing, batch operations, and compliance scope

The sign and verify endpoints hand you a signature without handing out the private key. A webhook receiver can check the payload really came from your service. An audit chain can show entries were not edited after the fact. A license file can be checked offline by anyone holding the public half. Batch encrypt and decrypt keep the same guarantees while cutting the API overhead of a bulk migration job. Signing is its own policy path, so a token that may sign cannot decrypt as a side effect.

Tokenization keeps the mapping between token and real value inside Vault, so everything downstream carries an opaque string that is worthless on its own. That is what shrinks PCI scope (the Payment Card Industry rules that apply to any system touching card data) and PII scope, because the token is not the data and the system holding it drops out of the audit boundary. Pair tokenization with an encrypt-only frontend and you have a shape auditors recognize on sight: the sensitive value never lands in the application database in readable form.

terminal
vault write transit/sign/orders input=$(echo -n "payload-hash" | base64)
vault write transit/verify/orders input=$(echo -n "payload-hash" | base64) signature=$SIG
output
signature vault:v1:8sK2...
valid true
# private key never left Vault — verify on receipt
terminal
vault write transit/tokenize/orders plaintext=$(echo -n "4111-1111-1111-1111" | base64)
vault write transit/detokenize/orders token="t:8xYz..."
output
token t:8xYz...
# store token in app DB, not PAN
plaintext NDExMS0xMTExLTExMTEtMTExMQ==
# detokenize only from batch/compliance role

Review Transit policy separately from your KV access lists (ACLs, the rules saying which token may read which secret). The two go stale in different ways. An encrypt-only path left behind by a service you decommissioned last quarter is untidy but harmless. Decrypt on that same key is a live route to the data that nobody is watching. In a Transit review the question is not who can read secrets. It is who can decrypt.

Transit keys do not replicate between clusters on their own. Each Vault cluster holds its own key material, so a blob encrypted in eu-west will not open in us-east unless you planned for it, either with a replicated key or with one global Transit mount that both regions call. Settle the region boundaries before your apps start encrypting locally and shipping blobs across the world. Ciphertext stranded in the wrong region is a slow, miserable thing to unpick.

When an auditor asks who can decrypt production data, the answer is a list of Vault policy paths and the identity names from the audit log. "The app uses Transit" is not an answer. If you cannot produce that list in a few minutes, decrypt is wider than you believe and the bulk-decrypt alarms are almost certainly not wired up yet.

Write down every Transit key name, the team that owns it, and the operations that key allows, in the same catalog you already keep for secret engines. Keys with no owner quietly collect decrypt grants that nobody thinks to strip out when someone leaves.

terminal
vault policy read orders-frontend
vault policy read orders-batch
vault token capabilities transit/decrypt/orders
output
path "transit/encrypt/orders" { capabilities = ["update"] }
path "transit/decrypt/orders" { capabilities = ["update"] }
# frontend token: encrypt only — batch token: decrypt allowed

The split is the whole point: keys in Vault, ciphertext in your database. A stolen database backup is no longer enough to read customer fields, because the thief still needs a Vault token with decrypt on that key. Keep the key version with the blob, which Vault's ciphertext prefix already does for you, so a rotation never strands rows that nothing knows how to open.

Give each service a short-lived token whose policy allows transit/encrypt/payments and transit/decrypt/payments and nothing else. No updating the key, no exporting it. Exportable keys exist for migrations off Vault, and export deserves the same handling as break-glass: two people, a ticket, and an alarm that fires when it happens.

Backfilling an existing column is where batch input earns its keep. One HTTP request per row will take days and hammer Vault the whole time. Send blocks of rows instead. Measure the round trip time (RTT, how long a call takes to reach Vault and come back) from the region your app runs in, then either move Vault closer to the encrypting services or accept that tax on every write.

Try this

Create a Transit key, encrypt a sample string, rotate the key, then decrypt the value you saved before the rotate. Watch the prefix on new ciphertext move to v2 while the old v1 value still opens.

terminal
vault secrets enable transit
vault write -f transit/keys/payments
vault write transit/encrypt/payments plaintext=$(echo -n '[email protected]' | base64)
vault write -f transit/keys/payments/rotate
vault write transit/decrypt/payments ciphertext=vault:v1:...
output
Key Value
--- -----
ciphertext vault:v1:8SDd3WHDOjf7gl...
Key Value
--- -----
latest_version 2
Key Value
--- -----
plaintext YWxpY2VAZXhhbXBsZS5jb20=
# base64 -d -> [email protected] (v1 still works after rotate)

Takeaway

The habit worth taking from Transit is that a data key never lives in application config. Convergent encryption, signing and the batch endpoints are options you switch on when a specific problem asks for them. The version prefix on the ciphertext is the small detail that makes rotating a production key something you can do on a Tuesday afternoon.

Next, pick one sensitive column. Route new writes through Transit today, then plan the backfill with batch encrypt and a decrypt-on-read path, so old rows and new rows both work while the migration is running.

Encrypt-only counts for nothing if decrypt is handed out freely
Precise access control is the reason Transit is worth the API round trip, so do not give that away by granting decrypt broadly. Frontends get encrypt, and sign if they need it. Decrypt belongs to the one batch job or service that genuinely needs readable data, and that path should be rate-limited. An app with unlimited decrypt is a slow data exfiltration tool for whoever compromises it. Alarm on bulk-decrypt volume as well, because from Vault's side that spike is exactly what a breach looks like.
Quick check
01Why does a Transit ciphertext start with a version marker like vault:v1:?
Correct — The version travels with the data, which is what lets you re-encrypt lazily instead of all at once.
Incorrect — Keeping environments apart is a job for separate keys and policy, not for the prefix.
Incorrect — Transit handles symmetric keys too; the prefix is about which key version, not which algorithm.
Incorrect — The prefix is a label, not compression.
02Rewrap is different from decrypting and re-encrypting in your own code because rewrap…
Incorrect — Neither the key nor the plaintext leaves Vault during a rewrap.
Correct — That is why rewrap is the safe way to migrate data in bulk during a rotation.
Incorrect — Rewrap is a Transit operation, and it acts on ciphertext blobs.
Incorrect — Old versions stop working when you raise min_decryption_version, once the rewrap has finished.
03What should a frontend service normally be allowed to do with Transit?
Incorrect — Wide decrypt turns one compromised frontend into a way to read the whole database.
Incorrect — Managing keys is admin work, not something an application token should carry.
Correct — Write-only encryption keeps stored data safe even when the frontend is taken over.
Incorrect — Root-equivalent access on the mount throws away least privilege entirely.

Related