Encrypting Kubernetes Secrets at rest in etcd

Kubernetes Secrets are only base64 by default. Turn on an EncryptionConfiguration and rotate the key without downtime.

A Kubernetes Secret is not encrypted — it is base64-encoded, which is not the same thing. Anyone who can read etcd directly (a backup, a snapshot, a compromised control-plane node) can decode every password, token, and TLS key in the cluster. The fix is an EncryptionConfiguration on the API server: Secrets are encrypted with a provider key before they are written to etcd, and decrypted transparently on read.

Turning encryption on is a two-step job: configure the provider, then rewrite existing Secrets so they pass through the encrypting API server. Skipping the rewrite leaves plaintext Secrets sitting in etcd until someone touches them. If you are new to how Secrets reach pods, start with the Kubernetes administration track before hardening etcd.

Prove base64 is not encryption

Before enabling encryption, look at what etcd actually stores. The value is readable with a one-liner — that is what an attacker with etcd access gets.

Turn on encryption at rest

Create an EncryptionConfiguration with secretbox as the active provider and identity last so the API server can still read existing plaintext Secrets during rollout. Point the API server at the file with --encryption-provider-config and restart.

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources: [secrets]
    providers:
      - secretbox:
          keys:
            - name: key1
              secret: <32-byte-base64-key>
      - identity: {}   # read old plaintext during rollout

Encrypt what already exists

New Secrets encrypt automatically. Existing ones stay plaintext in etcd until rewritten. The standard approach is a replace loop across all namespaces — each Secret is read (decrypted by apiserver), then written back (re-encrypted).

Rotating the encryption key without downtime

Add the new key first in the provider list, restart the API server, rewrite all Secrets again, then remove the old key on a second pass. Secrets encrypted with the old key are re-encrypted with the new one during the rewrite. Never remove a key before every Secret has been rewritten — you will lose decryptability.

Encryption at rest does not help if someone with RBAC get secrets can read them through the API — that is working as designed. Pair etcd encryption with tight RBAC, audit logging on Secret access, and consider External Secrets so the authoritative copy lives outside the cluster entirely.

# Step 1: add key2 BEFORE key1, restart, rewrite all secrets
providers:
  - secretbox:
      keys:
        - name: key2
          secret: <new-32-byte-key>
        - name: key1
          secret: <old-key>
  - identity: {}
 
# Step 2: after rewrite completes, drop key1
providers:
  - secretbox:
      keys:
        - name: key2
          secret: <new-32-byte-key>
  - identity: {}

Where this goes next

Encryption at rest protects etcd backups. RBAC protects who can read Secrets through the API. External Secrets Operator or Vault Kubernetes auth removes long-lived bootstrap tokens entirely. The Kubernetes security (CKS) path covers etcd encryption alongside the rest of the control-plane hardening checklist.

Related posts