How secrets leak

Logs, ps, crash dumps, child processes, repos.

In short. How secrets leak in practice: logs, process listings, crash dumps, child processes, and git repos.

A secret is any string that grants access to whoever presents it: a database password, an API key, a cloud access token, a TLS private key. Think of it as a front-door key with one crucial difference — it's text. Copying it is free, instant, and invisible, and every system that touches it can keep a copy without you noticing. Most secret leaks are not breaches in the Hollywood sense. Nobody picks a lock. Someone pastes a key somewhere convenient, the copy outlives the moment, and eventually the wrong person reads it. This lesson maps where those copies end up and why attackers find them so fast.

What counts as a secret

Three terms, precisely. A credential is anything used to prove identity. A secret is a credential whose entire security rests on staying unknown — passwords, API keys, signing keys, connection strings. A token is a machine-issued secret, usually a random string like ghp_... (GitHub) or AKIA... (an AWS access key ID), often with a recognizable prefix precisely so scanners can spot it. Most of these are bearer credentials: the system accepting them trusts whoever *bears* (presents) them, no further questions asked. Your production database cannot tell the difference between your deploy script and an attacker on another continent holding the same 40 characters.

That bearer property is the entire threat model. A leaked secret requires no exploit, no vulnerability, no malware — the attacker simply logs in, indistinguishable from you, bypassing every firewall and detection rule you've built, because the activity looks legitimate. The only true remediation is rotation: issuing a new secret and revoking the old one, so every stolen copy stops working at once.

Why leaks beat exploits

The scale is not hypothetical. GitGuardian's annual scan of public GitHub found roughly 24 million newly committed secrets in 2024 alone, and it reports that a large share of leaked secrets stay valid long after they are exposed. Discovery is fully automated: honeytoken studies — researchers planting deliberately fake AWS keys in public repos — show the first unauthorized use attempt typically arrives within minutes of git push. GitHub's secret scanning notifies providers fast, but attacker automation races it, and wins on anything the scanners don't recognize.

Real incidents follow the pattern. Uber's 2022 breach escalated when the attacker found PowerShell scripts on an internal share containing hardcoded admin credentials for the company's privileged-access system. The 2021 Codecov supply-chain attack worked by silently exfiltrating environment variables from thousands of CI pipelines — because CI is where everyone's secrets congregate. Neither required a software vulnerability in the victim's own code.

Where the copies land

The first and largest vector is source code. git commit makes a secret effectively permanent: delete the file and the value survives in history, in every clone, and in every fork — the next lesson covers that in depth. The vector people underestimate most, though, is container images. A Dockerfile ARG or ENV feels ephemeral. It isn't:

# Bake a token in with a build arg — feels temporary, isn't
docker build -t payments-api:1.4.2 \
  --build-arg NPM_TOKEN=npm_4kX9mQvTz7Jw2LhPbN6cRd8s .
 
# Every layer's metadata ships with the image
docker history payments-api:1.4.2 --no-trunc | grep NPM_TOKEN
# -> ARG NPM_TOKEN                                                        0B
# -> RUN |1 NPM_TOKEN=npm_4kX9mQvTz7Jw2LhPbN6cRd8s /bin/sh -c npm ci # buildkit   211MB

Anyone with pull access to the registry — which at many companies means every engineer, every CI runner, and any attacker who compromises either — can read that token with one command. Files are no better: COPY .env . bakes the file into a layer, where it persists even if a later step deletes it. The fix is BuildKit secret mounts (RUN --mount=type=secret,id=npm_token ...), which expose the value only during that single build step and never write it to a layer.

Next, the machine you're sitting at. Shell history and the process table are both readable long after a command finishes:

# Paste a token into a "one-off" command...
curl -s -H "Authorization: Bearer ghp_wK9xT2mQ8vL4jR7nB3cY5dF1aZ6sE0hU" \
  https://api.github.com/user
 
# ...and it outlives the request
tail -n 1 ~/.bash_history
# curl -s -H "Authorization: Bearer ghp_wK9xT2mQ8vL4jR7nB3cY5dF1aZ6sE0hU" https://api.github.com/user
 
# Long-running processes expose their arguments to every user on the host
ps -eo pid,args | grep [p]g_dump
#  4172 /usr/bin/pg_dump --dbname=postgresql://app:S3cretPw@db:5432/prod

Everything in ps output is world-readable on that host — pass secrets via environment variables or files, never as command-line arguments. Then come logs: applications that print their config at startup, CI jobs run with set -x (which echoes every command, arguments included), crash reporters that ship the full environment alongside a stack trace. Finally the human layer: secrets pasted into Slack, Jira tickets, and wiki pages, where search indexing makes them findable by anyone in the company — or by anyone who phishes a single account.

Find them before attackers do

Attackers scan continuously, so the only posture that works is scanning yourself with the same class of tooling, earlier in the pipeline. gitleaks is the standard open-source scanner:

# Scan the working tree; -v prints each finding inline
# (v8.19+ syntax — `gitleaks git .` walks full history too)
gitleaks dir . -v --report-path gitleaks-report.json
 
# Finding:     AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
# Secret:      AKIAIOSFODNN7EXAMPLE
# RuleID:      aws-access-token
# Entropy:     3.684184
# File:        deploy/staging.env
# Line:        3
# Fingerprint: deploy/staging.env:aws-access-token:3
#
# 4:12PM INF scanned ~842 KB (862341 bytes) in 214ms
# 4:12PM WRN leaks found: 1

gitleaks dir scans a directory's current contents; gitleaks git walks a repository's full commit history, which is where most surprises live. Its cousin trufflehog goes further and *verifies* findings by calling the provider's API to check whether the credential is still live — invaluable for triage, since any repo with years of history surfaces plenty of long-dead keys. Run a scanner in three places: as a pre-commit hook (cheapest — the secret never leaves the laptop), in CI on every push, and on a schedule across the whole organization.

Shrink the blast radius

Prevention will never be perfect, so design for the leak that gets through. Prefer short-lived credentials over static ones: cloud OIDC federation lets a CI job exchange a signed identity token for short-lived cloud credentials — typically valid for an hour or less — so there is no long-lived key to leak in the first place. Scope every secret to the minimum it needs — a leaked read-only token is an incident; a leaked admin token is a company-wide crisis. And keep an inventory: you cannot rotate a secret you don't know exists.

One vector deserves a lesson of its own. Git's core design goal is to never lose data — which means it also never loses your secrets. Deleting a file, force-pushing, even deleting the repository from the server can still leave the value recoverable from clones, forks, and reflogs. Next, we dig into exactly how secrets persist in git history, how to prove whether one ever entered yours, and what real cleanup requires.

Related