>_SecOpsLog_
CoursesFluxSecrets: SOPS & Sealed Secrets

Secrets: SOPS & Sealed Secrets

Encrypted secrets in Git.

Advanced12 min · lesson 10 of 12

Git is a ledger. Every commit is permanent, copied to every clone, and readable by anyone who can pull the repo. That is exactly what you want for your Kubernetes manifests, and exactly what you do not want for a database password. GitOps (running your cluster from a Git repository as the single source of truth for what runs) means your secrets have to live in that repo too. The whole game is putting them there without putting them in plaintext, so the copy in Git is useless to everyone except the one cluster meant to run it.

Why plaintext in Git never comes back

A secret committed once in plaintext is a secret leaked forever. Delete the file in the next commit and it still sits in the history, in every fork, on every laptop that ran git clone, and in whatever backup your Git host keeps. Once that happens, rotating the credential is the only real fix, and rotating a live credential across a running system is slow and error-prone. So the rule is blunt: no readable secret ever touches a commit. Flux decrypts one of these formats, SOPS, on its own. The other two, Sealed Secrets and External Secrets, lean on a helper controller you run next to Flux. Three patterns, and we will take them one at a time.

SOPS: lock the values, keep the labels

SOPS (Secrets OPerationS, a Mozilla-born tool) works like a jewelry box with a clear label on the lid. It encrypts the values in a YAML or JSON file while leaving the keys, the structure, and the field names readable. So stringData.password stays visible as a field name, but its value turns into a block of ciphertext. That buys you two things. You can review a diff and see which secret changed without seeing the secret, and Flux can still parse the file as a Kubernetes object before it decrypts anything inside it.

SOPS does not invent its own master-key cryptography. It hands that job to a backend: age, PGP (Pretty Good Privacy), or a cloud KMS (Key Management Service) such as AWS KMS, GCP KMS, or Azure Key Vault. age (a small, modern file-encryption tool) is the simplest, and the one most Flux setups start with. You generate a keypair once.

terminal
$ age-keygen -o age.agekey
output
Public key: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
age.agekey
# created: 2026-07-20T10:14:22Z
# public key: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
AGE-SECRET-KEY-1GFPYD4YQ2K9V7XWZ3M8N6TQ2L5RJ8HZ4C0P9WK7D3F6A2S1E4UQ5MXVT2

The public key, the string starting with age1, is the padlock. Anyone can hold it. It only locks. The private key, starting with AGE-SECRET-KEY-1, is the only thing that decrypts it, and it never goes into Git. You encrypt against the public key, and you decide once, in a small config file, which fields get encrypted.

.sops.yaml
creation_rules:
  - path_regex: .*\.yaml$
    encrypted_regex: ^(data|stringData)$
    age: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p

That encrypted_regex line is the one that matters. It tells SOPS to encrypt only the data and stringData values and leave everything else alone, so apiVersion, kind, and metadata stay in the clear. Without it, SOPS encrypts the whole file, and Flux can no longer tell the object is even a Secret. With the config in place, encryption is one command.

terminal
$ sops --encrypt --in-place db-credentials.yaml
$ cat db-credentials.yaml
output
apiVersion: v1
kind: Secret
metadata:
    name: db-credentials
    namespace: default
type: Opaque
stringData:
    password: ENC[AES256_GCM,data:0aQ9vX7r,iv:9Jm2kP==,tag:kf3Rw2==,type:str]
sops:
    age:
        - recipient: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
          enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBQZ2sx...
            -----END AGE ENCRYPTED FILE-----
    lastmodified: "2026-07-20T10:20:01Z"
    mac: ENC[AES256_GCM,data:Tm9uY2U==,iv:Qk1==,tag:Lp7==,type:str]
    encrypted_regex: ^(data|stringData)$
    version: 3.9.4

The password is now ENC[AES256_GCM,...], encrypted with AES-256 in Galois/Counter Mode (a standard authenticated cipher). The sops block at the bottom records who can decrypt it (the age recipient), a message authentication code (a MAC, which lets you detect if anyone edited the ciphertext), and the SOPS version. This file is safe to commit. Anyone reading the repo sees the shape of your secret and none of its contents.

Handing Flux the private key

Flux's kustomize-controller (the component that reads your Git repo and applies manifests) is what decrypts SOPS files. To do that, it needs the private key somewhere it can reach: stored as a Kubernetes Secret in the cluster. You create that Secret from the age key, then point your Kustomization at it with a decryption block.

clusters/prod/apps.yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: apps
  namespace: flux-system
spec:
  interval: 10m
  path: ./apps
  prune: true
  sourceRef:
    kind: GitRepository
    name: flux-system
  decryption:
    provider: sops
    secretRef:
      name: sops-age
terminal
$ cat age.agekey | kubectl create secret generic sops-age \
    --namespace=flux-system \
    --from-file=age.agekey=/dev/stdin
output
secret/sops-age created

Two details decide whether this works at all. The Secret has to live in the same namespace as the Kustomization, flux-system here, because that is where the controller looks. And the data key inside the Secret has to end in .agekey for age (or .asc for PGP), because that suffix is what the controller scans for. Get the suffix wrong and decryption fails quietly rather than loudly.

Now the loop is closed. Ciphertext sits in Git. The private key sits in one Secret in one namespace. On each reconcile, the controller pulls the repo, decrypts the SOPS files in memory, and applies the resulting plaintext Secret to the cluster. The plaintext never returns to Git and never lands on disk. If you would rather the raw key never sit in the cluster, point decryption.provider: sops at a cloud KMS instead. Flux then calls out to the KMS using the cluster's workload identity, and the key stays locked inside the KMS.

Sealed Secrets: seal it to one cluster

Sealed Secrets (from Bitnami) flips the arrangement. Instead of you holding the key, a controller inside the cluster holds it, and you seal each secret to that specific controller. Think of a night-deposit slot at a bank: anyone can push an envelope in, only the branch with the vault key can take one out. You run a small tool called kubeseal on your laptop, give it the controller's public certificate, and it produces a SealedSecret that only that controller can open.

terminal
$ kubeseal --format yaml \
    --controller-name sealed-secrets-controller \
    --controller-namespace kube-system \
    < db-credentials.yaml > db-credentials-sealed.yaml
$ cat db-credentials-sealed.yaml
output
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: db-credentials
  namespace: default
  creationTimestamp: null
spec:
  encryptedData:
    password: AgBvA9k2m1Qz7RtY0oXpN6cLwFq8s...t0KpQ==
  template:
    metadata:
      name: db-credentials
      namespace: default
      creationTimestamp: null
    type: Opaque

kubeseal fetches the controller's public certificate over the API (or you pass it with --cert for an air-gapped machine), encrypts each value, and writes a SealedSecret custom resource (a CRD, Custom Resource Definition, is an object type an add-on installs into Kubernetes). Flux applies that resource like any other manifest. The controller watches for SealedSecret objects, decrypts them with its private key, and creates the matching plain Secret in the cluster. By default the seal is strict: it is bound to one name and one namespace, so you cannot copy a sealed value into a different namespace and have it open. There are looser namespace-wide and cluster-wide scopes, and each step looser is a step you should be able to justify.

Where the key lives and who can steal it

The three approaches move the same puzzle piece to different places. With SOPS, you hold the key and Flux borrows it. With Sealed Secrets, the cluster holds the key and you seal to it. With the External Secrets Operator (a controller that pulls secrets from an outside store like HashiCorp Vault or AWS Secrets Manager), the secret never enters Git at all; only a reference does, and the operator's own credentials open the real store.

Same secret, three places to keep the key
SOPS
In Git
Secret with encrypted values
Key lives
age / KMS key you control
Decrypts
kustomize-controller at reconcile
Sealed Secrets
In Git
SealedSecret CRD
Key lives
controller in kube-system
Decrypts
sealed-secrets controller in-cluster
External Secrets
In Git
ExternalSecret reference only
Key lives
Vault / cloud secret store
Decrypts
operator syncs into a Secret
Ciphertext or a reference lives in Git; the unlocking key lives somewhere scoped and guarded.

Whichever you pick, the security boundary is one specific key or credential, not the repository. With SOPS, the age private key (or KMS access) is the crown jewel: whoever holds it decrypts every secret in the repo, past and present. With Sealed Secrets, the controller's private key in kube-system is the prize, and it also means a value sealed to a dead cluster cannot be unsealed by a fresh one unless you backed up and restored that key. With External Secrets, the blast radius is whatever the operator's Vault or cloud role is allowed to read.

The key is the entire boundary
A leaked age private key or over-broad KMS access decrypts every secret in the repo, including everything still in Git history. Store the key as a Secret that only kustomize-controller can read (or keep it in a KMS so the raw key never rests in-cluster), scope who can use it, and rotate on any exposure. Your secrets are safe because of that key's protection, not because they sit in Git.

Checking your blast radius

The question a defender actually asks is: if this key leaks, what opens? For SOPS, that reduces to who can read the sops-age Secret. Kubernetes RBAC (Role-Based Access Control, the rules for who can do what) decides it, and you can test the answer directly instead of guessing.

terminal
$ kubectl auth can-i get secret/sops-age -n flux-system \
    --as=system:serviceaccount:default:web-app
output
no

no is the answer you want: your application service accounts have no path to the decryption key. Anything other than no, for an account that should not have it, is a finding to chase down. Next, confirm decryption is actually happening on reconcile, because a broken key shows up as a stuck Kustomization, not a crash.

terminal
$ flux get kustomizations
output
NAME         REVISION            SUSPENDED  READY  MESSAGE
flux-system  main@sha1:a1b2c3d4  False      True   Applied revision: main@sha1:a1b2c3d4
apps         main@sha1:a1b2c3d4  False      True   Applied revision: main@sha1:a1b2c3d4

READY True with an applied revision means the controller decrypted and applied cleanly. When the key is wrong or missing, you get the opposite, and the message points at where it broke.

terminal
$ flux get kustomizations apps
output
NAME  REVISION            SUSPENDED  READY  MESSAGE
apps  main@sha1:a1b2c3d4  False      False  kustomize build failed: failed to decrypt db-credentials.yaml: age: no identity matched any of the recipients
Quick check
01Your SOPS-encrypted Secret sits in a public repo and a stranger clones the whole history. What is actually keeping the password out of their hands?
Incorrect — Git stores your file byte for byte as you committed it. Disk encryption on the host protects the server, not someone who already holds a clone.
Incorrect — The rule in .sops.yaml covers data and stringData only, so kind and metadata stay readable and the object is easy to identify.
Correct — Ciphertext without the matching key is noise. Protect that one key and the repository itself can be as public as you like.
Incorrect — The controller reads from Git and decrypts in memory during reconcile. It never writes anything back to your branch.
02Your .sops.yaml sets encrypted_regex: ^(data|stringData)$. Delete that line and SOPS encrypts the whole file instead. What breaks first?
Correct — Keeping the labels in the clear is what lets the controller parse the object before it decrypts, and lets a reviewer see which secret moved.
Incorrect — Size is not the issue. age handles a whole small YAML file happily, so the regex is protecting you from something else.
Incorrect — Left alone, SOPS encrypts every key in the file. Limiting it to two fields is a decision you make in config, not a limit of the tool.
Incorrect — Placement comes from the Secret's own metadata. The encryption rule only decides which fields get locked.
03flux get kustomizations apps shows READY False with 'failed to decrypt db-credentials.yaml: age: no identity matched any of the recipients'. Which mistake fits that exact wording?
Incorrect — A source that cannot be fetched fails earlier and reports a source problem. This run got far enough to attempt decryption.
Incorrect — A plaintext Secret would apply without complaint. The wording here proves decryption ran and came up with no matching identity.
Incorrect — That mistake reads as the decryption Secret being missing entirely. Here it was found and opened, and held nothing usable.
Correct — Only a suffix of .agekey, or .asc for PGP, marks a data key as an identity. Any other name leaves the controller empty handed.

Two things belong in your runbook. Set RBAC so only kustomize-controller's service account can read the decryption Secret, and re-check it with kubectl auth can-i after every RBAC change. And treat any exposure of the age key or the Sealed Secrets controller key as a full rotation event: generate a new key, re-encrypt or re-seal every secret in the repo, and rotate the underlying credentials themselves, because a copy of the old ciphertext plus the old key still opens everything.

Try this

Run age-keygen -o age.agekey on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: the key is the entire boundary. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Multi-tenancy & RBACHardening Flux & the supply chain

Related

What Atlantis is: PR-driven TerraformAtlantisGitOps & what Argo CD isArgo CDArgo CD vs FluxComparisonGitOps interview questionsInterview prep