KMS, HSM & key hierarchies

Envelope encryption, seal-wrap, PKCS#11, FIPS, and key ceremonies.

Underneath every secrets manager is a key-management story, and at the expert tier you need to reason about it directly: what a KMS actually protects, why hardware (an HSM) matters for the top of the hierarchy, and how key hierarchies keep a single root compromise from decrypting the world. This is the layer auditors and regulators care about, and the layer that turns "we encrypt secrets" into a defensible statement.

Key hierarchies: why nobody encrypts data with the root

Serious systems never encrypt data directly with a top-level key. Instead they build a hierarchy: a root key (in an HSM) wraps key-encryption keys (KEKs), which wrap data-encryption keys (DEKs), which encrypt the actual data. Each layer limits blast radius and enables rotation at the right granularity — rotate a DEK to re-protect one dataset, rotate a KEK to force re-wrap of many DEKs, and keep the root essentially immortal and offline. This is the same envelope pattern as the cloud managers, generalized: the higher the key, the fewer operations it performs and the more it is protected.

HSMs, PKCS#11, and seal-wrap

A Hardware Security Module is a tamper-resistant device that generates and uses keys without ever exporting them — you ask it to sign or unwrap, and the private key never leaves the hardware. Software talks to HSMs through the PKCS#11 standard (or a cloud HSM/KMS API). For Vault specifically, seal-wrap uses an HSM/KMS to add an extra layer of encryption around especially sensitive stored values (and is required for some compliance regimes). The reason to care: an HSM turns "an admin could copy the key" into "the key physically cannot leave," which is the assurance regulated environments demand.

seal "pkcs11" {
  lib            = "/usr/lib/softhsm/libsofthsm2.so"
  slot           = "0"
  pin            = "env://HSM_PIN"
  key_label      = "vault-hsm-key"
  hmac_key_label = "vault-hsm-hmac"
}
# the master key is wrapped by a key that never leaves the HSM.
# FIPS 140-2/3 validated HSMs are how regulated shops satisfy "keys in hardware".

BYOK, key ceremonies, and rotation

Two governance patterns round this out. Bring-Your-Own-Key (BYOK) imports key material you generated into the cloud KMS/HSM, so you can attest to how it was created and, in some models, retain the ability to withdraw it — a control regulated customers often require. A key ceremony is the auditable, multi-person process for generating and backing up a root key: split custody, witnesses, a recorded script, tamper-evident storage. Both exist because at the top of the hierarchy, the process around the key is as important as the key — you are protecting the thing that protects everything else.