CoursesCheckov & IaC scanningRunning it as a program

Running it as a program

Policy, exceptions, metrics.

Advanced12 min · lesson 12 of 12

Running checkov -d . once is a surprise kitchen inspection. Someone walks in, writes down what is wrong that morning, and leaves. Running Checkov as a program is the food hygiene regime behind the inspection: the same rules in every kitchen, every exception written down with a date on it, and numbers that tell you whether the kitchens are getting cleaner. The command itself barely changes. What changes is that policy, exceptions and metrics become artefacts you own and version, instead of arguments you have again in every repository.

Three things hold a scanning program up. First, policy written as code: a .checkov.yaml file plus a shared pack of custom checks that every repository pulls. Second, a suppression ledger, meaning a reviewable list of every skip, each with a ticket against it and a date to look at it again. Third, trend metrics built from archived JSON (JavaScript Object Notation, the machine-readable results file) and SARIF (Static Analysis Results Interchange Format, the file that code-scanning dashboards read). Take away any one of the three and the scanning turns into noise people scroll past.

One definition of passing, everywhere

If every team decides for itself what a passing scan looks like, you do not have a program, you have twelve opinions. A .checkov.yaml at the root of the repository, or one shared file passed with --config-file, settles it in writing: which frameworks are in scope, which check IDs sit under hard-fail-on and which under soft-fail-on, which skip-check waivers are allowed, and whether output is compact. Those two lists are the dial you turn as the organisation gets better, and turning the dial is your decision, not a judgement each engineer makes at midnight.

terminal
$ checkov -d . --config-file .checkov.yaml --external-checks-dir ./org-policies --compact
output
Passed checks: 850, Failed checks: 12, Skipped checks: 31
Check: CKV_AWS_20: "S3 Bucket has public access block"
FAILED for resource: aws_s3_bucket.new_uploads
# hard-fail-on IDs in config caused exit=1

Keep exceptions where people can read them

Every grown-up program grants exceptions. The immature ones grant them invisibly. An inline skip comment (the mechanism covered in cv-suppress) should carry a ticket reference, so anyone reading the file knows who accepted the risk and why. A global skip-check in the config is a much heavier thing: it silences that check across the whole repository, including code nobody has written yet. So put the grep for checkov:skip on a schedule. If the skip count grows faster than the fix count, the program is losing ground.

terminal
$ grep -rn "checkov:skip" --include="*.tf" --include="*.yaml" . | grep -v "JIRA-" | wc -l
$ grep -rn "checkov:skip" --include="*.tf" . | wc -l
output
3
47
Checkov program governance loop
1Policy as code
.checkov.yaml + org packs
2Scan every change
same tiers, all repos
3Exception review
ticketed, time-boxed waivers
4Metrics loop
trends justify tightening
Metrics earn you the right to tighten policy, which raises the bar for the next scan.

Measure the trend, not the moment

A single scan tells you about one commit on one afternoon. A program needs a line on a graph. Archive the JSON and SARIF output for every commit, then track four numbers over time: failed checks, skips, entries left in the baseline file, and how long a finding takes to get fixed. Do not put the raw failure total on a slide as the headline number. The day you onboard a legacy repository that total jumps, and the jump means you finally looked, not that anything broke overnight.

terminal
$ checkov -d . --config-file .checkov.yaml \
--output json --output sarif \
--output-file-path results
$ jq 'if type=="array" then .[].summary else .summary end' results/results_json.json
output
{
"passed": 812,
"failed": 37,
"skipped": 9,
"parsing_errors": 0,
"resource_count": 214,
"checkov_version": "3.2.451"
}

Ship the rules like a library

Custom checks travel best as a versioned package. Pull them with --external-checks-git, or vendor them as a submodule under org-policies/, and pin every repository's CI (continuous integration, the automated pipeline that runs on each push) to the same tag. Pinning is what makes a new org-wide rule land on the next pipeline run rather than whenever somebody remembers to copy a file across.

terminal
$ checkov -d . --external-checks-git github.com/acme/checkov-policies?ref=v2026.07.1 \
--config-file .checkov.yaml -o json --output-file-path results
output
Check: CKV_ACME_1: "Ensure resources have a cost-center tag"
FAILED for resource: aws_s3_bucket.logs
Wrote JSON output to results/results_json.json
Severity gating quietly does nothing without a platform key
Write hard-fail-on: [HIGH, CRITICAL] in open-source Checkov and the build stays green no matter what fails. Built-in checks carry no severity rating unless the Prisma Cloud platform supplies one through --bc-api-key, so the list matches nothing and the gate fails open. Until that enrichment is wired up, name the check IDs you want to block explicitly.

Route findings to whoever owns the thing

A finding that lands in a shared inbox belongs to nobody. Route it instead: tag Terraform resources with Owner, map Kubernetes namespaces to teams, or split repositories so that a CODEOWNERS file (the list GitHub reads to pick required reviewers for a path) puts .checkov.yaml and the policy packs in front of the right people. Programs die quietly when every failure emails the same distribution list and everyone on it assumes someone else is reading it.

Every waiver gets an expiry date

An accepted risk is a decision with a shelf life. Write the ticket and the review date into the skip reason itself, then run a grep audit each quarter and delete what has expired. A skip that was reasonable in January can be indefensible by June, and the file will not tell you that unless the date is sitting in it. Treat a global skip-check in the config as an organisation-wide waiver: it goes through architecture review, never through someone trying to turn a red build green before lunch.

The two lists are how you roll a rule out without a revolt. Put CKV_AWS_20 and CKV_AWS_23 under hard-fail-on today, because nobody should be merging a public bucket. Leave CKV_AWS_18 access logging under soft-fail-on while teams work through it, so the finding is reported and counted but blocks nobody. One caution worth repeating: those lists want check IDs. A severity list looks like it is working and does nothing, so explicit IDs are the backbone in open-source mode.

Numbers that survive contact with a leadership meeting: baseline entries going down, the median number of days between a check failing and someone fixing it, the share of repositories running the pinned policy pack, and suppressions with no ticket ID heading toward zero. The number that misleads everyone is the raw failed-check total in the week you import a legacy monorepo (one repository holding many projects). That spike is adoption, not a fire.

The loop that scales looks like this. Central security owns org-policies/ and cuts a tag. Application repositories pin that tag in CI. When a developer wants a new rule, they open a pull request against org-policies with test fixtures attached, security reviews and merges it, a new tag goes out, and each repository bumps its pin on its own schedule. Compare that with every team forking the same YAML into a private gist, and you can see which of the two you can still audit a year from now.

Tighten on a cadence. Once a quarter, move one check from soft-fail-on to hard-fail-on, and only when the metrics say teams have the capacity to fix it. That is the ratchet. Announce it in Slack with the check IDs and their Guide links two sprints before the config change lands, because the fastest way to lose goodwill is a red build nobody was warned about.

Keep the config file next to the code so a laptop and the pipeline reach the same verdict. When it lives in the repository, checkov -d . on a branch fails for exactly the reasons the pipeline will fail, and nobody meets a new rule for the first time during a release. When the policy lives only inside the CI job definition, local runs are advisory fiction and every tightening arrives as a surprise.

Read the two grep numbers together. Forty-seven skips across the Terraform files is not a scandal on its own. The three with no ticket-shaped reason are where you start asking questions, because those are the ones nobody can defend. Track the pair every quarter. Total skips rising while the untraceable three stay near zero is a team accepting risk out loud. Both numbers rising is tribal knowledge piling up, and you will meet it again during an incident.

Archiving is mechanical, so make it boring. Save both output files against the commit that produced them, keep SARIF for the dashboard and JSON for scripts, and never re-run an old scan to answer a question about last month. One shape gotcha: a multi-framework run writes a list of result objects rather than a single one, which is why the jq filter (jq is the command line JSON filter) tests the type before reaching for .summary. Skip that guard and your metrics job breaks the first time someone adds Kubernetes manifests to the repo.

A reference .checkov.yaml to copy

Security publishes one file and everyone else starts from it. Frameworks in scope. CKV_AWS_20 and CKV_AWS_23 under hard-fail-on. CKV_AWS_18 under soft-fail-on for the length of the logging rollout. skip-check entries only with a linked architecture decision record (an ADR, the short document that records why a decision was made and by whom). compact: true and download-external-modules: true as the sensible defaults. Application repositories inherit by copy or git submodule, and drift becomes visible the moment one of them is missing hard-fail-on.

terminal
# .checkov.yaml
framework:
- terraform
- kubernetes
hard-fail-on:
- CKV_AWS_20
- CKV_AWS_23
soft-fail-on:
- CKV_AWS_18
compact: true
download-external-modules: true
output
# Committed beside code — same policy on laptop and CI
# hard-fail-on uses check IDs, not HIGH/CRITICAL, in OSS mode

Deciding which tier a check belongs in is the judgement call worth spending real time on. A hard-fail check that teams cannot fix this sprint blocks everyone and turns into a ticket asking to switch the scanner off. A soft-fail check that stays soft forever is theatre: reported on every run, fixed by nobody. A rule of thumb that holds up. Hard-fail what has a short fix and a large blast radius, such as a public bucket or an admin IAM pattern you can name by check ID. Soft-fail what needs a migration.

Treat the policy pack as production code, because that is what it is. Fixtures proving the check catches the bad resource and leaves the good one alone, pytest (the Python test runner) running in the pack's own pipeline, a human review on every change, and semantic version tags so consumers know when an upgrade will make more builds fail. A pack shipped without tests eventually blocks a merge because of a bug in the rule, and that costs more trust than the rule was ever going to earn.

Adoption needs a number too. Pinning to v2026.07.1 gives you something countable: how many repositories are on the current tag, how many sit two tags behind, which ones never pinned at all. Chase the stragglers with a list rather than a memo, and expect the last few to be repositories where the config was copied once and quietly edited since. That is exactly the drift a shared file exists to expose.

Three failure modes account for most dead programs. Findings routed to a distribution list instead of an owner, so the queue never drains. Waivers with no expiry, so January's shortcut becomes permanent architecture. And a gate configured on severity in open-source mode, which reports plenty and blocks nothing while everyone believes it is enforcing. All three look healthy on a dashboard, which is why the suppression ledger and the pinned pack version tell you more than the failure count does.

Try this

Do this: spend thirty minutes treating policy like code. Run a scan against the shared config with the org pack loaded, archive a JSON summary from a second run, then count how many skips carry no ticket-shaped reason. Those four commands are the seed of a governance dashboard.

terminal
$ checkov -d . --config-file .checkov.yaml --external-checks-dir ./org-policies --compact --quiet
$ checkov -d . --config-file .checkov.yaml -o json --output-file-path results
$ jq 'if type=="array" then .[].summary else .summary end' results/results_json.json
$ grep -rn "checkov:skip" --include="*.tf" . | grep -vE "JIRA-|SEC-|TICKET-" | wc -l
output
Passed checks: 850, Failed checks: 12, Skipped checks: 31
{
"passed": 850,
"failed": 12,
"skipped": 31,
"resource_count": 240
}
3
# three skips lack ticket-shaped reasons — audit candidates

Takeaway

Remember: the CLI is only the enforcement arm. What makes scanning stick across forty repositories is one published .checkov.yaml, a policy pack with a version tag, and waivers that carry a ticket and a review date. Get those three artefacts under source control and the argument about what counts as passing stops happening in pull request comments.

Your next move here is operational. Put the quarterly skip audit in a calendar with a named owner, pin every repository to the current org-policies tag, and choose now the one check you will move from soft-fail-on to hard-fail-on next quarter, so you can announce its ID two sprints before the config change lands.

Quick check
01Which metric actually shows the program improving over time?
Incorrect — one snapshot jumps the week you import a legacy repo, which says nothing about posture.
Correct — burn-down plus remediation speed is the direction of travel leadership should see.
Incorrect — a count of installed versions measures packaging, not security posture.
Incorrect — codebase size grows with the product and tracks nothing about risk.
02Why insist that every suppression reason carries a ticket ID?
Incorrect — Checkov accepts any reason, even an empty one; the discipline is yours to enforce.
Correct — a grep plus a ticket trail turns scattered skips into exceptions someone approved.
Incorrect — expiry comes from your quarterly grep audit, not from anything the CLI does.
Incorrect — the platform key is about severity enrichment and has nothing to do with tickets.
03Your .checkov.yaml sets hard-fail-on: [HIGH, CRITICAL], the scan reports failures every run, but no build has ever gone red and there is no platform key in play. What do you do next?
Incorrect — no severity name matches, because built-in checks carry no severity in open-source mode; a longer list changes nothing.
Incorrect — reporting without blocking is the behaviour you already have, so this changes nothing except the wording of the config.
Correct — check IDs are what an open-source run can actually match, which restores the gate and the non-zero exit.
Incorrect — that throws away frameworks, tiers and waivers, and still leaves you with no gate.

Related