Key backends: age & KMS

age, AWS/GCP/Azure KMS, PGP.

Every SOPS file is locked twice, and only one of those locks is yours to pick. SOPS (Secrets OPerationS, started at Mozilla, now maintained by the getsops project inside the CNCF, the Cloud Native Computing Foundation) makes a brand-new random key for every single file, scrambles the values with it, then seals a copy of that key for each recipient you named. Choosing a key backend means choosing who holds those sealed copies and what they have to prove before one opens. Get it wrong and a stolen laptop reads your production database password. Get it right and the same laptop gets a permission denied, and a human can go and read that refusal in an audit log afterwards.

One Data Key, Many Wrappers

A toolbox in a shared workshop makes the shape of this obvious. One small key opens the box. Instead of handing out copies of that key on the street, you seal a copy in an envelope addressed to each person who should have access, and tape the envelopes to the lid. Anyone who can open their own envelope gets the key, and the key opens the box. The proper name for this is envelope encryption, and SOPS does it to every file it touches.

Concretely: SOPS generates a random 256-bit data key (32 bytes straight from the operating system's random source), encrypts each value with AES-256-GCM (Advanced Encryption Standard, 256-bit key, Galois/Counter Mode, a cipher that hides the content and detects tampering with it), and feeds the field's full path in the document into the encryption as AAD (additional authenticated data, extra text that is not hidden but is still covered by the tamper check). That path binding is why you cannot lift the ciphertext out of staging.db.password, paste it in as prod.db.password, and expect it to decrypt. The data key itself is then encrypted once per master key you configured, and those wrapped copies go into the sops: block at the foot of the file. Your keys stay readable in Git. Your values do not.

db:
    host: db.acme.internal          # not secret, left readable
    password: ENC[AES256_GCM,data:9f2aQ1t7Xc8=,iv:Yy0kR3vJ8s1pQm4d...,tag:1QdFqA==,type:str]
sops:
    kms:                            # wrapper 1: an AWS KMS key
        - arn: arn:aws:kms:eu-west-1:111122223333:key/1a2b3c4d-5e6f-7081-92a3-b4c5d6e7f809
          created_at: "2026-07-22T09:14:03Z"
          enc: AQICAHhwm0YaISJeR6nQ7Vd0Yy8pK...Zk9Q==
          aws_profile: ""
    age:                            # wrapper 2: an age recipient
        - recipient: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
          enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBB...
            -----END AGE ENCRYPTED FILE-----
    lastmodified: "2026-07-22T09:14:03Z"
    mac: ENC[AES256_GCM,data:Vv8kP2r...,iv:...,tag:...,type:str]
    unencrypted_suffix: _unencrypted
    version: 3.13.2
# That really is the whole block: since 3.10, SOPS leaves out
# the backend lists it is not using instead of writing empty ones.

Two things fall straight out of that layout. Adding or removing a backend is cheap, because SOPS re-wraps the same small data key and never re-encrypts your values, which is why sops updatekeys finishes in a blink on a file holding a thousand secrets. And the wrappers are an OR, not an AND. Any single one of them recovers the data key, and the data key opens everything. Your secrets are exactly as well protected as the weakest recipient on that list, so a convenient key added in a hurry quietly lowers the bar for the whole file. The mac line is a message authentication code, a short fingerprint of the file's values computed with the data key, so hand-editing a ciphertext or dropping a field makes SOPS fail loudly instead of handing you garbage.

age: a Key File You Hold

age is the minimalist option, and it behaves like the house key on your keyring. (The name is pronounced "ah-gay". Filippo Valsorda designed it.) age-keygen hands you two short strings: a public recipient starting age1... that you paste into config and commit without a second thought, and a private identity starting AGE-SECRET-KEY-1... that you guard. No expiry dates, no web of trust, no keyservers, no config file. Underneath sit X25519 (an elliptic-curve key agreement, the maths that lets two parties derive a shared secret over a public channel) and ChaCha20-Poly1305 (a fast cipher that also authenticates what it encrypts), but the whole user-facing surface is those two strings.

# generate a keypair; this is the path SOPS checks by default
mkdir -p ~/.config/sops/age
age-keygen -o ~/.config/sops/age/keys.txt
 
# encrypt to that recipient (subcommand form, sops 3.9 and later)
sops encrypt --age age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p \
  secrets.yaml > secrets.enc.yaml
 
# decrypt: no recipient needed, SOPS reads the wrappers out of the file itself
sops decrypt secrets.enc.yaml

Public key: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
db:
    host: db.acme.internal
    password: pR0d-p4ss-9f2a

SOPS looks for your private identity in one fixed place per platform: $XDG_CONFIG_HOME/sops/age/keys.txt when that variable is set, otherwise $HOME/.config/sops/age/keys.txt on Linux, $HOME/Library/Application Support/sops/age/keys.txt on macOS, and %AppData%\sops\age\keys.txt on Windows. Environment variables override all of that, and the differences between them matter in a pipeline. SOPS_AGE_KEY_FILE points at a path on disk. SOPS_AGE_KEY carries the key material itself, so a CI (continuous integration) runner never writes it to a filesystem at all. SOPS_AGE_KEY_CMD, new in 3.10, names a command SOPS runs to fetch the key at the moment it is needed, which lets you pull it from a password manager or a hardware-backed helper instead of leaving it lying around. On the encrypt side, SOPS_AGE_RECIPIENTS saves you retyping recipients on every command.

Version 3.10 also widened what counts as an age identity, and two of those additions buy real security rather than convenience. Plugins are supported, so a recipient like age1yubikey1... backed by age-plugin-yubikey keeps the private half inside a hardware token that will not export it. Your laptop can decrypt. A forensic image of your laptop's disk cannot. SSH (Secure Shell) keys work too, both as recipients (ssh-ed25519 AAAA...) and as identities through SOPS_AGE_SSH_PRIVATE_KEY_FILE or SOPS_AGE_SSH_PRIVATE_KEY_CMD, which helps when a host already has a key you trust and you would rather not distribute a second one. Passphrase-protected identities landed in the same release, so an on-disk key can at least demand something you know.

# a CI runner that never writes the identity to disk (3.10 and later)
export SOPS_AGE_KEY_CMD="vault kv get -field=identity secret/ci/sops-age"
 
# pull a single field instead of the whole document
sops decrypt --extract '["db"]["password"]' secrets.enc.yaml
 
# is this file actually encrypted? filestatus prints JSON and exits 0 either way,
# so a pre-commit hook has to read the field, not the exit code (3.9 and later)
sops filestatus secrets.enc.yaml
sops filestatus secrets.yaml
sops filestatus secrets.yaml | grep -q '"encrypted":true' || echo "REFUSING TO COMMIT PLAINTEXT"

pR0d-p4ss-9f2a
{"encrypted":true}
{"encrypted":false}
REFUSING TO COMMIT PLAINTEXT

Cloud KMS: a Key You Can Borrow, Never Hold

A cloud KMS (Key Management Service) is the bank vault version, where you never receive a key at all. You push your sealed envelope through a slot, a clerk checks who you are, opens it behind the counter, hands back the contents, and writes your name and the time in a ledger. AWS KMS, Google Cloud KMS and Azure Key Vault all follow that shape. The master key is created inside an HSM (hardware security module, tamper-resistant hardware built so that key material cannot be exported) and never leaves it. SOPS holds no master key whatsoever. On encrypt it sends the data key to the provider's Encrypt API (application programming interface, the provider's remote function call) and stores whatever comes back; on decrypt it sends the wrapped copy to Decrypt and receives the data key over TLS (Transport Layer Security, the encryption behind https).

That swap changes the question you have to keep answering. "Where is the key file and who has a copy?" turns into "which identities hold kms:Decrypt on this key ARN (Amazon Resource Name, the unique id AWS gives every resource), and what does the log say they did with it?" Three things follow. Access is granted and taken away centrally through IAM (Identity and Access Management), with no key file to chase around laptops. Every decryption writes a CloudTrail, Cloud Audit Logs or Azure Monitor event carrying a caller identity, a timestamp and a source address, so you can alert on a decrypt by a principal that has no business decrypting. And you get a genuine kill switch: disable the key and every decrypt stops, everywhere, at once.

# AWS KMS: the ARN of a symmetric key.
# Append +<role ARN> and SOPS calls sts:AssumeRole first, then talks to KMS as that role.
sops encrypt --kms 'arn:aws:kms:eu-west-1:111122223333:key/1a2b3c4d-5e6f-7081-92a3-b4c5d6e7f809+arn:aws:iam::111122223333:role/sops-prod' \
  prod-secrets.yaml > prod.enc.yaml
 
# Google Cloud KMS: a resource id, authenticated by Application Default Credentials
sops encrypt --gcp-kms projects/acme-prod/locations/europe-west1/keyRings/sops/cryptoKeys/sops-key \
  prod-secrets.yaml > prod.gcp.enc.yaml
 
# Azure Key Vault: vault URL, key name AND key version. All three are required;
# SOPS rejects a URL that stops at the key name.
sops encrypt --azure-kv https://acme-sops.vault.azure.net/keys/sops-key/8a1f0f5e0e1a4d7fb63a5b1d9a2c4e77 \
  prod-secrets.yaml > prod.az.enc.yaml
 
# confirm which key SOPS actually recorded (yq reads YAML on the command line)
yq '.sops.kms[0].arn' prod.enc.yaml

arn:aws:kms:eu-west-1:111122223333:key/1a2b3c4d-5e6f-7081-92a3-b4c5d6e7f809

Two AWS details earn their keep. The first is encryption context. --encryption-context 'Environment:production,App:checkout' attaches key/value pairs that KMS itself authenticates: the identical pairs must be presented on decrypt or the call fails, and you can write an IAM condition on kms:EncryptionContext:Environment so a staging role is refused when it reaches for a production file. Those pairs are not secret. They sit in the sops: block in plain sight, and the protection comes from the policy, not from hiding them. In .sops.yaml you can only set a context per key inside a key_groups block, which is one reason that longer form is worth learning. The second detail is scope. SOPS calls exactly two operations on your KMS key, Encrypt and Decrypt, plus sts:AssumeRole (Security Token Service, the API that hands out temporary credentials) when you use the +role form. It never calls kms:GenerateDataKey or kms:DescribeKey, because it makes the data key locally with its own random source, so those permissions in the policy you copied off a blog post are dead weight. A deploy identity that only reads secrets needs kms:Decrypt and nothing else. On GCP the equivalent split is roles/cloudkms.cryptoKeyDecrypter for consumers and roles/cloudkms.cryptoKeyEncrypter for authors; on Azure it is the Key Vault Crypto User role, or the encrypt and decrypt key permissions if you are still on access policies.

# does the key work, and did anyone notice?
sops decrypt prod.enc.yaml > /dev/null && echo "decrypt ok"
 
# the same call seen from the KMS side. CloudTrail lags by a few minutes,
# so an empty result right after the decrypt does not mean nothing was logged.
aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=EventName,AttributeValue=Decrypt \
  --max-results 1 \
  --query 'Events[0].{user:Username,when:EventTime}' --output json

decrypt ok
{
    "user": "dana",
    "when": "2026-07-22T09:16:41+00:00"
}

The cost is a dependency you cannot wish away. Every encrypt and every decrypt is a network call to the provider with valid credentials attached, so a plane with no wifi, a pipeline without cloud credentials, or a regional outage all mean no secrets today. The nastier version is circular: if the account holding the key is the thing that broke, the runbook you encrypted with that key is unreadable at exactly the moment you need it. The kill switch cuts both ways too. Disabling the key locks out your own deployments, and a scheduled key deletion in AWS runs a waiting period of 7 to 30 days and then destroys the key for good, taking every file ever encrypted to it along with it. Nothing recovers from that.

PGP, and Where It Still Fits

PGP (Pretty Good Privacy), driven through GnuPG (GNU Privacy Guard), is the drawer of old house keys in the hallway. It was the first backend SOPS shipped with, it is still fully supported, and it is fiddlier than either of the others. You encrypt to a 40-character key fingerprint, and SOPS asks your local gpg-agent to unwrap the data key, falling back to running the gpg binary itself. SOPS_GPG_EXEC points it at a different binary when your distribution keeps gpg somewhere odd, and SOPS_PGP_FP sets default fingerprints, mirroring what SOPS_AGE_RECIPIENTS does for age.

Two reasons it survives. Hardware: a private key living on a smartcard or a YubiKey gives you the same "a copy of the disk is useless" property that an age plugin does, and PGP has offered it for well over a decade. Portability: a fingerprint works in every cloud and in none of them, which suits teams who refuse to let their break-glass path depend on one provider being reachable. The costs are keyring management, expiry dates that break decryption on a Tuesday morning with no warning, and an agent that behaves differently inside a container than it does on your desktop. Watch subkeys especially. Handing SOPS a subkey id does not guarantee GnuPG encrypts to that subkey; a trailing ! forces it, and SOPS only passes that marker through intact from 3.9.3 onward.

# the full 40-hex-character fingerprint SOPS expects, with the spaces stripped out
gpg --list-keys --with-colons [email protected] | awk -F: '/^fpr:/ {print $10; exit}'
 
# encrypt to it
sops encrypt --pgp 85D77543B3D624B63CEA9E6DBC17301B491B3F21 secrets.yaml > secrets.enc.yaml
 
# decrypting goes through gpg-agent; point SOPS at a different gpg if you must
SOPS_GPG_EXEC=/usr/bin/gpg2 sops decrypt secrets.enc.yaml | head -2

85D77543B3D624B63CEA9E6DBC17301B491B3F21
db:
    host: db.acme.internal

Two Backends on One File

Teams running SOPS at any scale usually land on two backends per production file: the cloud KMS key for everyday work, because it is audited and revocable, and one age recipient kept offline for the day the cloud is the outage. .sops.yaml is where that pairing gets written down so nobody has to remember it. One warning about the shape below. Every key inside a single key_groups entry is an OR, which is what you want here. Two separate groups mean something else entirely: SOPS splits the data key into Shamir shares (fragments that only rebuild the key once enough of them are combined), and then several groups have to succeed instead of one. Note also that the short kms: form in a creation rule takes a plain string, so the moment you need a role or a context you have to move to key_groups. Matching rules to paths is the next lesson's subject.

# Rules are matched top to bottom. The first one that matches wins, and only it applies.
creation_rules:
  # production: the cloud key for daily work, plus an offline break-glass age
  # recipient. One key_group, so any single one of them opens the file.
  - path_regex: secrets/prod/.*\.yaml$
    key_groups:
      - kms:
          - arn: arn:aws:kms:eu-west-1:111122223333:key/1a2b3c4d-5e6f-7081-92a3-b4c5d6e7f809
            role: arn:aws:iam::111122223333:role/sops-prod
            context:
              Environment: production   # KMS authenticates this on every decrypt
        age:
          - age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
 
  # everything else: developer age keys, no cloud round trip, works on a train.
  # The short form is one comma-separated string, and every key in a rule
  # written this way lands in a single group.
  - age: 'age1yt3tfqlfrwdwx0z0ynwplcr6qxcxfaqycuprpmy89nr83ltx74tqdpszlw,age1u79dtsm3xhqfhkjkgs2qkmv6xnv3n5qmqcdrfa5r9jknvtnq0e5s2vlfr2'

Files that already exist do not pick up a new rule on their own. sops updatekeys reads the current rules, works out the difference against what the file already carries, shows it to you, and re-wraps the existing data key for the new recipient list once you agree.

sops updatekeys secrets/prod/db.enc.yaml

2026/07/22 09:16:03 Syncing keys for file /home/dana/infra/secrets/prod/db.enc.yaml
The following changes will be made to the file's groups:
Group 1
    age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
+++ arn:aws:kms:eu-west-1:111122223333:key/1a2b3c4d-5e6f-7081-92a3-b4c5d6e7f809
Is this okay? (y/n):y
2026/07/22 09:16:07 File /home/dana/infra/secrets/prod/db.enc.yaml synced with new keys

Read that diff carefully, because it also shows what the command does not do. updatekeys rewrites the wrappers and leaves everything else alone. The data key inside is the same data key it was this morning, and not one encrypted value changes.

Reading a Decrypt Failure

When decryption fails, SOPS reports every wrapper it tried and why each one turned it away. Below is a build runner whose role lost kms:Decrypt in a policy cleanup, and which has no age key file on it at all.

sops decrypt secrets/prod/db.enc.yaml

Failed to get the data key required to decrypt the SOPS file.
 
Group 0: FAILED
  age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p: FAILED
    - | failed to load age identities: failed to open file: open
      | /home/runner/.config/sops/age/keys.txt: no such file or
      | directory
 
  arn:aws:kms:eu-west-1:111122223333:key/1a2b3c4d-5e6f-7081-92a3-b4c5d6e7f809: FAILED
    - | failed to decrypt sops data key with AWS KMS: operation
      | error KMS: Decrypt, https response error StatusCode: 400,
      | RequestID: 8b1f0a7c-2d55-4c19-9f3e-7a0c1de24bb1, api error
      | AccessDeniedException: User:
      | arn:aws:sts::111122223333:assumed-role/ci-deployer/gha is
      | not authorized to perform: kms:Decrypt on the resource
      | associated with this ciphertext because no identity-based
      | policy allows the kms:Decrypt action
 
Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

The age wrapper is reported first because SOPS tries age, then PGP, then everything else by default, an order you can change with --decryption-order or SOPS_DECRYPTION_ORDER. Two failures, two possible fixes, and picking between them is a security decision wearing a debugging costume. Restoring kms:Decrypt on the deploy role keeps every deployment in CloudTrail and lets you cut access again with a single policy edit. Copying the age identity onto the runner turns the build green in thirty seconds and creates a second, permanently unlogged path into production that nobody will remember exists in six months. Whichever you choose, test it as the identity that will actually do the work: aws sts assume-role into the deploy role, export the temporary credentials into a clean shell, and run sops filestatus and then sops decrypt there. Your own laptop credentials will cheerfully tell you everything is fine.

Try this

Run age-keygen -o ~/.config/sops/age/keys.txt on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: an age identity is a bearer token with no expiry. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related