Baselines & incremental adoption

Onboard a legacy repo sanely.

Buying a 20-year-old house, you do not walk away because the inspector finds knob-and-tube wiring. You write down what is already wrong, agree not to make it worse, and book the electrician for spring. Onboarding a legacy Terraform repo to Checkov works the same way. The first scan can come back with hundreds of failures. If that wall of red blocks every pull request on Monday, someone quietly deletes the CI (continuous integration, the automated build that runs on every change) step by Friday. A baseline lets you admit today's debt honestly without punishing the next person who touches the repo.

Suppose you flip the gate to hard-fail tomorrow morning. Every open pull request goes red under five years of accumulated debt, almost none of it written by the people now stuck behind it. A baseline is the controlled way to turn scanning on in production without declaring a company-wide incident on day one. Then you ratchet the debt down on purpose, one sprint at a time.

Here is the mechanic in one breath. --create-baseline photographs every failure you have right now and writes it to .checkov.baseline, a JSON (JavaScript Object Notation, a plain-text data format) file that acts as your debt register. You commit that file. Every later scan run with --baseline still evaluates everything, but only fails on findings that are missing from the photograph. That is a different instrument from an inline suppression, which is a single skip attached to one resource with a written reason beside it.

Snapshot the debt

Run this once against the legacy repo to record every failure it has today. --compact trims the per-check chatter so the snapshot run stays readable. The file lands next to the path you scanned. Commit it to git so CI and every laptop read the same debt register instead of each building a private one.

$ checkov -d . --create-baseline --compact
$ git add .checkov.baseline && git status

Baseline created with 847 findings.
 
Wrote baseline to .checkov.baseline
 
new file:   .checkov.baseline

Fail only on new findings

In CI, hand the committed baseline back to Checkov. A developer who adds a new unencrypted bucket gets blocked. The twelve unencrypted buckets that were already there do not fail the build. During a grace period you can add --soft-fail so new findings still print while the exit code stays zero. How that gate is wired into GitHub Actions is cv-cicd. The baseline is what keeps the gate from drowning developers in noise they inherited.

$ checkov -d . --baseline .checkov.baseline --compact
$ echo exit=$?

Passed checks: 812, Failed checks: 2, Skipped checks: 0
 
Check: CKV_AWS_21: "Ensure all data stored in the S3 bucket have versioning enabled"
	FAILED for resource: aws_s3_bucket.new_uploads
	File: /s3.tf:44-52
 
exit=1

Burning the debt down

A baseline is a ratchet, not a graveyard. Fix a batch of resources, regenerate with --create-baseline, then read the git diff before you commit it. Entries that disappeared are real wins. Entries that appeared are somebody accepting new debt under cover of a routine regeneration. Watch the number shrink month over month. That trend line is the program metric leadership actually understands (cv-program). Never grow the file quietly to make a red build go away.

$ checkov -d . --create-baseline --compact
$ git diff --stat .checkov.baseline

Baseline created with 801 findings.
 
 .checkov.baseline | 46 deletions(-)

Buying a grace sprint with soft-fail

Teams new to a scanning gate sometimes need a sprint to sort out their workflow before failures start blocking merges. --soft-fail prints the new findings and still exits zero, which buys exactly that sprint. Set an end date the day you turn it on. A gate that never bites teaches everyone to scroll past it.

$ checkov -d . --baseline .checkov.baseline --soft-fail --compact
$ echo exit=$?

Failed checks: 2
 
Check: CKV_AWS_21: ...
	FAILED for resource: aws_s3_bucket.new_uploads
 
exit=0

What is actually in the file

Each entry keys off three things: the file path, the resource address, and the check IDs that were failing, meaning the CKV_AWS_21 style identifiers Checkov gives every rule. Line numbers are deliberately left out, so adding a comment at the top of main.tf does not invalidate every entry below it. The file is plain JSON sitting in git, which means a pull request that grows it deserves the same scrutiny you would give a new suppression. Never baseline a critical finding you intend to fix. Baseline the debt that has a scheduled burn-down behind it. Here jq, a command-line tool for reading JSON, is enough to see the shape.

$ jq 'length' .checkov.baseline
$ jq '.[0:2]' .checkov.baseline

847
[
  {"file": "/main.tf", "resource": "aws_s3_bucket.legacy", "check_ids": ["CKV_AWS_18", "CKV_AWS_21"]},
  ...
]

That shape has a useful consequence. Because an entry names the exact check IDs that were failing, an old resource is forgiven for those checks and nothing else. Let the legacy bucket start failing an ID that was not in its entry, whether because someone edited it or because an upgrade shipped a new rule, and it fails the build like any other finding. The baseline pardons specific known sins, not a resource for life.

Treat a regeneration like any other code review. Added rows mean someone accepted new debt. Removed rows mean someone fixed something. Do not regenerate in a panic in the middle of a Checkov upgrade without working out which failures come from new rules and which are regressions you wrote yourself. The release notes list the new CKV IDs, and those show up as fresh baseline misses on code nobody touched.

A baseline is not an amnesty for critical exposure. Plenty of programs baseline the LOW and MEDIUM debt while keeping CKV_AWS_20 public access as a hard failure even on legacy resources, listed under hard-fail-on in .checkov.yaml. Baseline for new debt only, plus a short list of checks you refuse to grandfather, is the arrangement most enterprises land on, and cv-cicd shows how it gets wired into Actions.

When two teams share one monorepo baseline, arguments about the file growing are really arguments about who accepted the debt. If ownership reads more cleanly that way, split the baselines per service directory or per Terraform root. One giant file hides which team signed off on which accepted failure.

The social contract in practice

The rollout order matters more than the tooling. Scan the legacy estate once and look at the number before you decide anything. Create the baseline on a clean commit so the snapshot reflects code that is actually merged, not whatever sits half-finished on your branch. Turn the gate on with --baseline from the first day rather than later, because a gate that arrives after the team has learned to ignore the scan is a much harder sell than one that was never noisy.

Choosing between a baseline and an inline skip comes down to volume and authorship. A skip is for one resource where you can write down a reason and a ticket, and where a reviewer sees that reason sitting next to the code. A baseline is for the eight hundred failures that predate everyone in the room, where writing eight hundred individual reasons would be theatre. Bulk history goes in the baseline. Deliberate one-off exceptions stay inline with a justification (cv-suppress).

The CI wiring lives in cv-cicd. What belongs here is the habit around it. After every remediation sprint, regenerate, read the diff, commit the shrinkage. A pull request that adds baseline rows with no linked ticket should fail human review even when CI is green, because growing the baseline is accepting debt, not paying it.

A baseline nobody regenerates goes stale in a very quiet way. Say you fix versioning on that legacy bucket but leave its old entry in the file. Six months later someone reverts the module, the bucket loses versioning again, and the stale entry still forgives it. Regenerating after a remediation sprint is what turns a fix into something that cannot silently come back.

Concretely: after the first full scan of a legacy estate, snapshot once with checkov -d . --create-baseline --compact. The JSON lists file path, resource address and failing check_ids, with no line numbers, so reformatting a file does not invalidate the entries inside it. Commit it. From then on every laptop and every CI job passes --baseline .checkov.baseline. A developer who adds aws_s3_bucket.new without versioning sees CKV_AWS_21 fail on their branch. The twelve legacy buckets recorded in the baseline stay quiet. That is the whole social contract: we stop making it worse while we fix what is already wrong.

Put regeneration on the calendar instead of reaching for it when a build goes red. Fix logging on a module used in six places, rerun the create step, look for deletions in the git diff, commit the shrink. If the diff surprises you, stop and find out why before you commit it. The surprise is the finding.

Try this

Do this: create a baseline, re-run with --baseline so you watch the old failures go quiet, then fix one resource and regenerate so you see the file shrink. That shrink is the burn-down signal you can put in front of leadership without translating anything first.

$ checkov -d . --create-baseline --compact
$ ls -la .checkov.baseline
$ checkov -d . --baseline .checkov.baseline --compact --quiet
$ checkov -d . --create-baseline --compact && git diff --stat .checkov.baseline

Baseline saved to .checkov.baseline
 
Passed checks: 812, Failed checks: 2, Skipped checks: 35
# only NEW failures print; baselined debt is silent
 
 .checkov.baseline | 40 -----
 1 file changed, 40 deletions
# after fixes, regeneration deletes resolved entries

Takeaway

Remember: a baseline is a ratchet, not a permanent pardon. It gets scanning onto a messy estate without failing every pull request on day one, and it still stops the next mistake at the pull request. The price is discipline. Regenerate after each remediation sprint, review added rows the way you would review a suppression, and never baseline the critical exposure you have already promised to fix this week.

Keep a short never-grandfather list beside the baseline, public S3 buckets being the standard entry, so the ratchet has a floor it cannot slip below. Then go compare scanners in cv-landscape, but only once your baseline and triage habits exist. Run a bake-off before that and you are counting whose tool prints more red, nothing else.

Related