Scanning, VEX & re-scan

Gate on what matters; catch new CVEs later.

Advanced30 min · lesson 11 of 15

08:00, and the security channel is on fire. A new critical CVE (Common Vulnerabilities and Exposures, the shared public identifier for one security flaw) has landed against a library your fleet ships. The real question is not 'is that library somewhere in our source code'. It is narrower and much harder: which of the artifacts actually running in production carry the bad version, and in those, does anything reach the broken code? A supermarket handles a peanut recall by reading the ingredient labels it already printed, not by re-testing every jar on every shelf. You work the same way. You never touch the running images. You scan the SBOMs (software bills of materials, the ingredient lists you generated and signed at build time). A scanner reads that list, checks every component against a vulnerability database, and prints exactly where you are exposed, usually buried inside a wall of findings that are present but cannot be exploited. This lesson is about turning that wall into a gate you can live with.

Scan the ingredient list, not the running image

Grype and Trivy can both point straight at a container and scan it. The practice that holds up over years is to scan the SBOM instead, the CycloneDX or SPDX document you produced in the previous lesson, using grype's own input scheme: grype sbom:./sbom.cdx.json. The SBOM is frozen and offline, so you can scan it a hundred times without pulling a single byte of image, and you get the same answer every time. The matching itself is unglamorous bookkeeping. Grype's vulnerability database is a compiled index assembled from NVD (the National Vulnerability Database, the US government's public advisory feed), GitHub Security Advisories, and each Linux distribution's own security tracker. Every entry is keyed by package name, ecosystem, and the range of versions a flaw affects. Grype turns each component in your SBOM into a purl (package URL, a standard string that names one package precisely) and a CPE (Common Platform Enumeration, the older naming scheme NVD uses), then asks one plain question: does this installed version fall inside any advisory's affected range? Every yes becomes one row. The --fail-on flag turns that report into a gate by setting the severity at which grype exits non-zero.

scan the stored SBOM, gate on critical
# Match the frozen SBOM against grype's current vulnerability DB,
# and fail (non-zero exit) if anything critical remains unaddressed.
grype sbom:./sbom.cdx.json --fail-on critical -o table
echo "exit: $?"
grype output — 1 unaddressed critical, gate fails
✔ Vulnerability DB [no update available]
✔ Scanned for vulnerabilities [42 vulnerability matches]
├── by severity: 1 critical, 6 high, 20 medium, 13 low, 2 negligible
└── by status: 9 fixed, 33 not-fixed, 0 ignored
NAME INSTALLED FIXED-IN TYPE VULNERABILITY SEVERITY
zlib1g 1:1.2.13.dfsg-1 deb CVE-2023-45853 Critical
libc-bin 2.36-9 2.36-9+deb12u3 deb CVE-2023-4911 High
libexpat1 2.5.0-1 2.5.0-1+deb12u1 deb CVE-2024-45490 High
libssl3 3.0.11-1~deb12u2 3.0.13-1~deb12u1 deb CVE-2024-0727 Medium
... (38 more rows)
[0000] ERROR discovered vulnerabilities at or above the severity threshold
exit: 1

The table is honest, it is noisy, and it fails your build. Two distinctions carry all the weight here. libc-bin and libssl3 have something in the FIXED-IN column. Those are real work items, and you close them by bumping the base image. The zlib1g critical, CVE-2023-45853, is the textbook false positive. The flaw lives in MiniZip, a contrib component that Debian does not compile into the zlib1g package it ships, so the vulnerable code is not in your artifact at all. Break the pipeline over that row and you teach your team a lesson you did not intend. The gate cries wolf, somebody loosens --fail-on until it stops tripping, and after that nobody reads the report. Flags get you part of the way there. --only-fixed drops findings that have no patch available, --fail-on sets the threshold. Neither severity nor fixability can express the sentence you actually need: the code is on disk, and nothing can reach it. That claim needs a separate document, portable and machine-readable, authored by a person willing to put their name on it.

VEX: say out loud why a finding does not apply

A VEX document (Vulnerability Exploitability eXchange) is a short note that answers one question about one vulnerability against one product: does this actually affect you? It behaves like the allergen card on a factory line. The ingredient sits in the warehouse; the card records whether any of it went into this batch. OpenVEX is the purl-native, openly specified version of that format, and it is what the cloud-native supply-chain tools read. Every statement carries a status: not_affected, affected, fixed, or under_investigation. When the status is not_affected, you also have to pick a justification from a closed list: component_not_present, vulnerable_code_not_present, vulnerable_code_not_in_execute_path, vulnerable_code_cannot_be_controlled_by_adversary, or inline_mitigations_already_exist. That closed list is the entire design. You cannot write 'we looked and it's fine'. You have to name an engineering reason a colleague can argue with. vexctl create writes the file. You author it under your security team's identity, and you scope it to one exact artifact. For the zlib finding the honest answer is not_affected with vulnerable_code_not_present, because MiniZip was never compiled in.

author the OpenVEX not_affected statement
vexctl create \
--author "Acme Security <[email protected]>" \
--product "pkg:oci/api@sha256:9f2c1e..." \
--vuln "CVE-2023-45853" \
--status "not_affected" \
--justification "vulnerable_code_not_present" \
--file vex.openvex.json
vex.openvex.json (the machine-readable assertion)
{
"@context": "https://openvex.dev/ns/v0.2.0",
"@id": "https://openvex.dev/docs/public/vex-2e6c1f9d0b...",
"author": "Acme Security <[email protected]>",
"timestamp": "2026-07-16T14:22:03.918273Z",
"version": 1,
"statements": [
{
"vulnerability": { "name": "CVE-2023-45853" },
"products": [
{ "@id": "pkg:oci/api@sha256:9f2c1e..." }
],
"status": "not_affected",
"justification": "vulnerable_code_not_present"
}
]
}

Every field in that file is doing work, so read it one line at a time. @context pins the OpenVEX schema version, so any tool that opens the document parses it the same way. vulnerability.name is the CVE. products[].@id is a purl, and it points at pkg:oci/api@sha256:9f2c1e…, an image digest rather than a floating tag like :latest. That one choice is the whole game. A statement bound to a digest describes one build and nothing else. Grype ties it back by comparing that product identifier against the subject your SBOM describes, and the CVE against the findings it produced. When both line up and the status is not_affected, grype moves the match from 'reported' to 'ignored' instead of deleting it, so the decision stays on the record where an auditor can find it. Suppressions also go stale. Commit the VEX next to the SBOM, review it in a pull request like any other code, and re-issue it with each release rather than carrying it forward forever.

Two re-scans: one for the VEX, one for tomorrow

Two kinds of re-scan matter, and teams mix them up constantly. The first applies the VEX to data that has not changed. grype … --vex vex.openvex.json folds the not_affected statement in, the zlib critical leaves the failing set, and --fail-on critical now exits 0, so the gate blocks on real risk only. Add --show-suppressed and the finding still prints, tagged '(suppressed by VEX)', which lets an auditor see exactly what was waved through and on what grounds. The second re-scan is about time passing. Your SBOM is immutable. The vulnerability database is not, because grype rebuilds it daily as new advisories are published. Scanning yesterday's SBOM against today's database is how you learn that an artifact that was clean when you built it is vulnerable this morning, with no rebuild and no redeploy, only a query against fresh data. That is the reason you keep old SBOMs at all: the CVE feed moves, the ingredient list does not. Scanning tells you which known-bad versions you already shipped. The next lesson, dependency integrity, is about stopping the bad or outright malicious version from entering the build in the first place.

re-scan with the VEX applied, gate now passes
grype sbom:./sbom.cdx.json \
--vex vex.openvex.json \
--fail-on critical --show-suppressed -o table
echo "exit: $?"
grype output — critical suppressed, exit 0
✔ Vulnerability DB [no update available]
✔ Scanned for vulnerabilities [41 vulnerability matches]
├── by severity: 0 critical, 6 high, 20 medium, 13 low, 2 negligible
└── by status: 9 fixed, 32 not-fixed, 1 ignored
NAME INSTALLED FIXED-IN TYPE VULNERABILITY SEVERITY
zlib1g 1:1.2.13.dfsg-1 deb CVE-2023-45853 Critical (suppressed by VEX)
libc-bin 2.36-9 2.36-9+deb12u3 deb CVE-2023-4911 High
libexpat1 2.5.0-1 2.5.0-1+deb12u1 deb CVE-2024-45490 High
... (39 more rows)
exit: 0
Triaging a scanner finding
Grype reports a CVE against a component
from scanning the stored SBOM
fix available + reachable
gate fails the build
bump the dependency, rebuild, re-scan
no fix yet, reachable
time-boxed exception
mitigate, track, re-review on expiry
present but not exploitable
VEX not_affected
justify from the enum, scope to digest, suppress
clean today
re-scan tomorrow
same SBOM, newer CVE database
Every branch names a reason you can defend. 'Fail on everything' flattens all four into noise, and then someone turns the scanner off.
A VEX not_affected scoped too widely never expires
Scope the statement to a bare package purl (pkg:deb/zlib1g) or to a floating tag instead of the image digest, and grype keeps hiding that CVE on every build you ever ship, including the later release where you did start compiling the vulnerable code path in. A suppression that was honest for one artifact quietly covers the next one, and nobody notices, because the report looks clean. Pin every product to the exact artifact digest and re-issue the VEX with each release, so a not_affected call can never outlive the build it was reasoned about.
Quick check
01You write a not_affected statement for CVE-2023-45853 with justification vulnerable_code_not_present and scope it to pkg:oci/api@sha256:9f2c1e…, today's image digest. Three releases later, a refactor pulls MiniZip into the build. What does the scanner do with the new image?
Correct — A digest-scoped statement covers one artifact, so the new digest gets a fresh, honest scan and the critical comes back.
Incorrect — No. OpenVEX has no notion of a permanent global suppression; scope and version decide whether a statement still applies.
Incorrect — Justifications are never cached. Grype re-matches statements by product identifier on every run, and the product (the digest) is different now.
Incorrect — A statement whose product does not match is ignored rather than treated as an error, and the finding prints as normal.
02A stored SBOM (software bill of materials) passed grype with exit 0 on build day, and nobody has touched the file since. Two weeks later the same unchanged file reports a new critical. What changed?
Incorrect — An SBOM is a frozen inventory that never changes after it is written, which is exactly what makes re-scanning it reproducible.
Incorrect — Scanning sbom:./sbom.cdx.json never touches the image; the match runs offline against the document alone.
Correct — The ingredient list is fixed while the CVE feed keeps moving, so old SBOMs scanned against fresh data surface newly disclosed flaws with no rebuild.
Incorrect — --fail-on is whatever you typed on the command line and does not change by itself; the new row is a genuine advisory match.
03Grype reports a high-severity CVE (Common Vulnerabilities and Exposures) with nothing in the FIXED-IN column, and your team confirms the vulnerable code really is reachable in your service. No patched version exists yet. Which move matches the triage in this lesson?
Incorrect — The code is present and reachable, so no not_affected justification would be truthful; that status is for findings nothing can exploit.
Correct — Reachable with no fix available is the 'no fix yet, reachable' branch, managed with a tracked exception that expires rather than a suppression.
Incorrect — --only-fixed hides everything with no patch available, which would bury a genuinely reachable risk instead of managing it.
Incorrect — There is no fixed version to move to; bumping the base image closes findings that carry a FIXED-IN value, which this one does not.

Try this

Work through “Two re-scans: one for the VEX, one for tomorrow” yourself on a sandbox you can throw away, following the commands above in order. Then break one step deliberately and re-run, so you have seen the failure before it finds you.

Takeaway

The trap worth remembering here: a VEX not_affected scoped too widely never expires. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related