KMS & envelope encryption

CMK/CMEK, key policies, grants, and data-at-rest patterns.

A cloud KMS (Key Management Service, the managed service that guards your encryption keys) is not a warehouse you pour data into and pull back out. If it were, every byte you own would ride through one metered endpoint and you would pay for the trip. A better model is a locksmith. You ask for a fresh, disposable key. The locksmith cuts one, hands it over, and keeps a sealed copy you can never open yourself. You lock your data with the disposable key, shred your own copy, and hand the sealed copy back for storage. Later, if the locksmith still recognizes you, they open the sealed copy and lend the key back so you can unlock. That round trip is envelope encryption, and once it clicks it rewires how you think about rotation, revocation, and who can actually read your data.

Two keys and a wrap

Every cloud hands you a root key that lives inside a hardened box called an HSM (Hardware Security Module, a tamper-resistant appliance certified to standards like FIPS 140-3, where FIPS is the U.S. government's Federal Information Processing Standards program) and never leaves it in plaintext. AWS calls that root key a KMS key (older docs abbreviate the term customer master key as CMK); the ones you create and control are customer managed keys. GCP calls it a CryptoKey inside a key ring, and the same idea shows up through other services under the label CMEK (Customer-Managed Encryption Keys). Azure calls it a Key Vault key, optionally HSM-backed. Whatever the name, that root key is a KEK (Key Encryption Key), and its only job is to wrap other keys.

The key that actually scrambles your bytes is a DEK (Data Encryption Key): a short-lived 256-bit AES (Advanced Encryption Standard) key you use on your own host and then destroy. Wrapping means the KEK encrypts the DEK, so the wrapped DEK can sit out in the open right next to the ciphertext with nothing to hide. Two tiers, one cheap cryptographic call per object or per batch. That is the entire model, and it holds on all three clouds even though the commands are spelled differently.

The envelope on AWS

AWS is the only one of the three with a built-in envelope call. GenerateDataKey hands you the same DEK in two forms at once: a Plaintext copy to use this second, and a CiphertextBlob copy that only KMS can unwrap. You encrypt your payload locally with the plaintext copy, wipe it from memory, and store the wrapped blob beside the ciphertext. To read the data back you send the blob to Decrypt, which unwraps it only if your identity is allowed. Your bulk data never touches the service.

# One call returns the DEK in two forms: Plaintext to use now, CiphertextBlob to store.
aws kms generate-data-key --key-id alias/app-data --key-spec AES_256 \
  --query '{plain:Plaintext,wrapped:CiphertextBlob}' --output json > dk.json
cat dk.json

{
    "plain": "hQ2m9FZ8k...v3n8xT0dQ7k=",
    "wrapped": "AQIDAHiF8n2...Kk3mYm8v2w=="
}

Now use that key and read it back. You encrypt locally, destroy the plaintext DEK, and keep only the ciphertext and the wrapped blob. Months later the blob goes back to KMS, which returns the same key if, and only if, you are still allowed to unwrap it.

# Split the two forms out, encrypt the file locally, then shred the plaintext DEK.
jq -r .plain   dk.json | base64 -d > dek.raw
jq -r .wrapped dk.json | base64 -d > wrapped.bin
# AES-256 in GCM (Galois/Counter Mode), an authenticated cipher; see the openssl caveat below.
openssl enc -aes-256-gcm -in report.csv -out report.enc -K "$(xxd -p -c256 dek.raw)"
shred -u dek.raw dk.json          # keep report.enc + wrapped.bin, nothing else
 
# Read it back later: hand the wrapped blob to KMS, get the same DEK only if allowed.
aws kms decrypt --ciphertext-blob fileb://wrapped.bin --key-id alias/app-data \
  --query Plaintext --output text

hQ2m9FZ8k...v3n8xT0dQ7k=

You are left with report.enc and wrapped.bin. Neither is any use without the other, and neither is any use to an attacker who cannot call Decrypt on that key. The bulk file never left your host, so the size of the backup has no bearing on your KMS bill.

GCP and Azure mint their own data key

Step off AWS and GenerateDataKey disappears. Code that assumes it silently fails to port. GCP Cloud KMS gives you Encrypt and Decrypt directly, capped at 64 KiB (kibibytes) of plaintext per call, so the idiomatic pattern is to generate the DEK yourself from a secure random source, wrap it with a single gcloud kms encrypt, and do the bulk work locally. Google's Tink library packages this exact dance if you would rather not wire it by hand.

# Create a key ring, then a symmetric key that auto-rotates every 90 days.
gcloud kms keyrings create app-ring --location=us          # prints: Created keyring [app-ring].
gcloud kms keys create app-data --keyring=app-ring --location=us \
  --purpose=encryption --rotation-period=90d \
  --next-rotation-time=2026-10-20T00:00:00Z               # prints: Created key [app-data].
 
gcloud kms keys describe app-data --keyring=app-ring --location=us \
  --format='yaml(purpose, rotationPeriod, nextRotationTime, versionTemplate.algorithm)'

purpose: ENCRYPT_DECRYPT
rotationPeriod: 7776000s
nextRotationTime: '2026-10-20T00:00:00Z'
versionTemplate:
  algorithm: GOOGLE_SYMMETRIC_ENCRYPTION

That rotationPeriod: 7776000s is 90 days in seconds, which is how the API stores it. With the key in place, mint your own DEK from the kernel's random source, wrap it with one Encrypt call, and confirm that unwrapping gives back byte-identical material.

head -c 32 /dev/urandom > dek.raw            # your own 256-bit DEK from the kernel's secure random source
sha256sum dek.raw                            # remember this fingerprint
 
gcloud kms encrypt --key=app-data --keyring=app-ring --location=us \
  --plaintext-file=dek.raw --ciphertext-file=dek.wrapped
openssl enc -aes-256-gcm -in report.csv -out report.enc -K "$(xxd -p -c256 dek.raw)"
shred -u dek.raw                             # store report.enc + dek.wrapped
 
# Unwrap and fingerprint again: the same bytes come back.
gcloud kms decrypt --key=app-data --keyring=app-ring --location=us \
  --ciphertext-file=dek.wrapped --plaintext-file=- | sha256sum

9f2c4b7e8a1d0c3f5b6e2a9d4c7f1e8b0a3d6c9f2e5b8a1d4c7f0e3b6a9d2c5f  dek.raw
9f2c4b7e8a1d0c3f5b6e2a9d4c7f1e8b0a3d6c9f2e5b8a1d4c7f0e3b6a9d2c5f  -

Azure is the same shape with one wrinkle. An ordinary Key Vault key is asymmetric, a public and private key pair (RSA, named for its inventors Rivest, Shamir, and Adleman, or an elliptic curve). So you wrap the DEK with RSA-OAEP-256, which is RSA encryption plus a padding scheme called OAEP (Optimal Asymmetric Encryption Padding). Symmetric AES key wrap exists only on Managed HSM, Azure's single-tenant HSM tier. Either way the DEK is born on your side and Azure only ever sees the wrapped form.

# Create an HSM-backed RSA key (Premium vault) permitted only to encrypt/decrypt.
az keyvault key create --vault-name kv-prod-crypto --name app-cmk \
  --kty RSA-HSM --size 3072 --ops encrypt decrypt \
  --query 'key.kid' -o tsv

https://kv-prod-crypto.vault.azure.net/keys/app-cmk/6b1f0a3d9c8e47a2b5f01c3d7e9a2b4c

# No GenerateDataKey here either: mint the DEK, then wrap it with RSA-OAEP-256.
DEK=$(head -c 32 /dev/urandom | base64)
az keyvault key encrypt --vault-name kv-prod-crypto --name app-cmk \
  --algorithm RSA-OAEP-256 --value "$DEK" --data-type base64 \
  --query result -o tsv

Lp7Qk9dWv3nRt0aX...wrapped-DEK-base64url...bQ  # store this next to the ciphertext

Why not drop the data key and call the service's own Encrypt on every object? Because those direct calls cap plaintext hard, 4 KiB on AWS and 64 KiB on GCP, and they push every byte through a rate-limited, metered endpoint with a network hop per object. Envelope encryption keeps one wrapped DEK per object or batch and runs the bulk work at local AES-GCM speed. That is why it is the default once you have real volume.

The key policy is the access control

Reading the ciphertext store is never enough to read the data. The caller also needs permission to unwrap the DEK, and that split is the whole security payoff. Someone who can list and download from your S3 bucket (Amazon's object store), Google Cloud Storage (GCS) bucket, or Azure Blob container, but who cannot call Decrypt, sees ciphertext and nothing else. You have turned 'can read the bucket' and 'can read the data' into two separate grants.

On AWS the key policy on the key is the source of truth. IAM (Identity and Access Management) policies can grant access only when that key policy allows it, so they add to the key policy, they never quietly replace it. For a programmatic caller, prefer a narrow grant that carries an encryption-context constraint, so one role can unwrap report DEKs but not payroll DEKs. One catch worth knowing: that constraint only bites if the DEK was generated with a matching encryption context, so pass --encryption-context team=reports to GenerateDataKey and the same value to Decrypt, and the grant's EncryptionContextSubset will hold.

# Grant one role Decrypt, only when the encryption context matches.
# create-grant is a management call: it takes the key ID or key ARN (Amazon Resource Name).
# Unlike encrypt/decrypt it will NOT accept an alias, so use the key ID behind alias/app-data.
aws kms create-grant --key-id 1234abcd-12ab-34cd-56ef-1234567890ab \
  --grantee-principal arn:aws:iam::123456789012:role/report-worker \
  --operations Decrypt \
  --constraints EncryptionContextSubset={team=reports} \
  --query '{GrantId:GrantId,GrantToken:GrantToken}' --output json

{
    "GrantId": "0d8f5a1c9b7e2a3f4d5e6c7b8a9f0e1d2c3b4a5f6e7d8c9b0a1f2e3d4c5b6a7f",
    "GrantToken": "AQpAM2RhZTk1MGMyNTk2ZmZmMzEyYWVhOWViN2I1YzgzNzQxYWJiNzRkMTU4OTI0YWU1MjM4NmEzODJmMGU0Zjc2KIAC...k4gGivedzFXo-dwN8fxjjq_ZZ9JFOj2ijIbj5FyogDCN0H4sIAAAA=="
}

That call returns a GrantId and a GrantToken. Grants are eventually consistent, so there is a short window, usually under five minutes, before the new permission is visible everywhere in KMS. Hand the grant token to the worker and it can decrypt right away, without waiting for the grant to finish spreading.

GCP gates the identical unwrap with a single role, roles/cloudkms.cryptoKeyDecrypter, which grants decrypt and nothing else. Tighten it with IAM Conditions, for instance an expiry timestamp so a service account loses access on a date you set. Conditions push the policy to version 3, which you can see in the response.

gcloud kms keys add-iam-policy-binding app-data \
  --keyring=app-ring --location=us \
  --member="serviceAccount:[email protected]" \
  --role=roles/cloudkms.cryptoKeyDecrypter \
  --condition='title=expires-2026-12-31,expression=request.time < timestamp("2027-01-01T00:00:00Z")'

Updated IAM policy for key [app-data].
bindings:
- condition:
    expression: request.time < timestamp("2027-01-01T00:00:00Z")
    title: expires-2026-12-31
  members:
  - serviceAccount:[email protected]
  role: roles/cloudkms.cryptoKeyDecrypter
etag: BwYf3kQZ1xY=
version: 3

Azure uses RBAC (Role-Based Access Control). The Key Vault Crypto User role carries the wrap, unwrap, encrypt, and decrypt actions, and you scope the assignment down to one key instead of the whole vault. Scoping by object id and principal type avoids a directory lookup that can fail on locked-down build agents.

VAULT_ID=$(az keyvault show --name kv-prod-crypto --query id -o tsv)
az role assignment create \
  --assignee-object-id 11111111-2222-3333-4444-555555555555 \
  --assignee-principal-type ServicePrincipal \
  --role "Key Vault Crypto User" \
  --scope "$VAULT_ID/keys/app-cmk"

{
  "createdOn": "2026-07-22T09:20:41.000000+00:00",
  "name": "9f8e7d6c-5b4a-3210-fedc-ba9876543210",
  "principalId": "11111111-2222-3333-4444-555555555555",
  "principalType": "ServicePrincipal",
  "roleDefinitionId": ".../roleDefinitions/12338af0-0e69-4776-bea7-57ae8d297424",
  "scope": ".../vaults/kv-prod-crypto/keys/app-cmk",
  "type": "Microsoft.Authorization/roleAssignments"
}

Audit who holds decrypt the way you would audit standing access to plaintext, because that is what it is. A principal with Decrypt plus read on the store has quiet, total reach into your data, and it will never surface in a review of bucket permissions alone.

Rotation without re-encrypting

Here is the part that catches people off guard. Only the wrapping key is versioned, so rotating it re-encrypts none of your stored data. New data keys get wrapped under the new key version while old versions stay around to unwrap old blobs. A petabyte of objects rotates for the price of zero bulk crypto.

AWS rotates once a year by default, or on a period you pick between 90 and 2560 days, and you can also trigger a rotation on demand. GCP rotates on the schedule you set at key creation, which you saw as rotationPeriod: 7776000s. Azure drives rotation from a policy attached to the key.

# Same rule as grants: these management calls take the key ID, not the alias.
# enable-key-rotation prints nothing; the status call shows what took effect.
aws kms enable-key-rotation --key-id 1234abcd-12ab-34cd-56ef-1234567890ab --rotation-period-in-days 180
aws kms get-key-rotation-status --key-id 1234abcd-12ab-34cd-56ef-1234567890ab

{
    "KeyRotationEnabled": true,
    "KeyId": "1234abcd-12ab-34cd-56ef-1234567890ab",
    "RotationPeriodInDays": 180,
    "NextRotationDate": "2027-01-18T14:32:07+00:00"
}

On Azure, describe the rotation as its own document and hand it to the key. This policy rotates the key 30 days before a 90-day expiry, both written as ISO-8601 durations (the P30D and P90D notation, where the leading P means 'period' and P90D means ninety days).

{
  "lifetimeActions": [
    {
      "trigger": { "timeBeforeExpiry": "P30D" },
      "action":  { "type": "Rotate" }
    }
  ],
  "attributes": { "expiryTime": "P90D" }
}

az keyvault key rotation-policy update \
  --vault-name kv-prod-crypto --name app-cmk \
  --value rotation-policy.json

{
  "attributes": {
    "expiryTime": "P90D",
    "updated": "2026-07-22T09:31:12+00:00"
  },
  "id": "https://kv-prod-crypto.vault.azure.net/keys/app-cmk/rotationpolicy",
  "lifetimeActions": [
    {
      "action":  { "type": "Rotate" },
      "trigger": { "timeBeforeExpiry": "P30D" }
    }
  ]
}

Revocation, and what it costs

Disabling the key is the opposite move, a reversible global kill switch. Every unwrap starts failing at once while your bulk ciphertext sits untouched, ready to read the second you re-enable. That is what an incident responder reaches for, not key deletion. On cost, KMS bills per key per month plus per cryptographic call, roughly one US dollar per key per month and three cents per 10,000 requests on AWS. Envelope encryption, plus caching one DEK across a batch, keeps that call count flat no matter how many rows you write. Keys are regional, so a multi-region design needs AWS multi-Region keys, a GCP multi-region location, or one Azure vault per region with replicated material.

So far the provider generated your master key and guarded it for you. Convenient, and for most workloads the right call. It does mean the cloud has, in principle, touched your root of trust. The next lesson pushes that boundary outward: BYOK (Bring Your Own Key), where you generate key material in your own HSM and import it under a formal key ceremony (a witnessed, audited procedure for handling raw key material), and HYOK (Hold Your Own Key), where the key never enters the cloud at all. Same envelope mechanics you ran here, a very different answer to the question of who you have to trust.

Try this

Run jq -r .plain dk.json | base64 -d > dek.raw on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: the wrapped DEK is the only copy. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related