CIS benchmarks with OpenSCAP

Scan, remediate, and track drift.

A home inspector walking through a house does not eyeball the wiring and offer an opinion. They carry a printed checklist: smoke detector in every bedroom, ground-fault outlets near the sink, a handrail on any stair with four or more steps. Each item is specific, and each one either passes or fails. Hardening a Linux server works best the same way. Instead of arguing about whether a box is "locked down," you check it against a written list of exact settings and get back a pass or fail for every one.

That written list is the CIS Benchmark (CIS is the Center for Internet Security, a non-profit that publishes free, consensus-built hardening guides for operating systems, databases, and cloud platforms). Every subject this course covers on its own is in that list: the Secure Shell (SSH) daemon config, PAM (Pluggable Authentication Modules, the stack that decides how a login is authenticated), sysctl kernel tunables, SELinux (Security-Enhanced Linux, the mandatory access control system built into the kernel), the audit daemon, mount options. Each one is a numbered control with a test and a fix. You could work through hundreds of them by hand. Or you point a scanner at the host and let it grade every control in one pass. OpenSCAP is that scanner, and it is the standard open-source one.

This lesson uses the Red Hat family (Red Hat Enterprise Linux, or RHEL, and its community rebuilds Rocky and Alma), where the packages are openscap-scanner and scap-security-guide and the content sits in /usr/share/xml/scap/ssg/content/. On Debian and Ubuntu the scanner ships in the libopenscap8 package and the CIS content in ssg-debderived, with the datastream under /usr/share/scap-security-guide/ (for example, ssg-ubuntu2204-ds.xml). Every oscap command from here on is identical across both.

What the scanner reads under the hood

OpenSCAP knows nothing about SSH or firewalls on its own. It is a generic engine that reads security content and runs it. That content follows SCAP (Security Content Automation Protocol, a family of standards for writing security checks that any tool can run). Two layers inside a SCAP bundle are worth naming. The checklist itself is written in XCCDF (Extensible Configuration Checklist Description Format), the human-readable list of controls, their severities, and how the total is scored, like the inspector's checklist. The actual test behind each control is written in OVAL (Open Vulnerability and Assessment Language), the machine-readable logic that goes and reads the real file or queries the running system to decide pass or fail, like the inspector's method for testing each item. The whole thing ships as one file called a datastream (the -ds.xml file), which is the inspector's binder holding the checklist and every test together.

One benchmark holds far more controls than any single host should apply, so the content groups them into profiles. Think of a profile as the edition of the checklist an inspector pulls for one kind of building: a named selection of rules, plus the exact values each rule checks for. CIS ships its controls in two levels. Level 1 is the safe baseline, settings that harden the box with little risk of breaking normal use. Level 2 is defense-in-depth for high-security environments, and some of its rules will turn off features you actually need. For a general server, start at Level 1 and treat Level 2 as a deliberate, tested upgrade. You can list every profile the content offers before you scan anything:

Run the first scan

Pick a profile and point oscap at the datastream. The --results file is the machine-readable record you keep and compare later; the --report file is a self-contained HTML page a human can read, with every control, its pass or fail, its severity, and the remediation text for anything that failed. The scan reads the live system, so run it with sudo.

The command streams a pass or fail for every rule to your terminal, but the overall grade lives in the results and the report. OpenSCAP scores the run from 0 to 100, weighted by rule. You do not have to open the HTML to read it; the number sits in the results XML, and one xmllint query pulls it out for logging or a trend graph. (CCE in that output is the Common Configuration Enumeration identifier, a stable catalog number for each control, handy when you cross-reference the benchmark PDF.)

Fix what failed, then prove it

OpenSCAP does not only find problems, it can write the fix. Point generate fix at your results file and it emits a remediation for every rule that failed, either as a Bash script or as an Ansible playbook (Ansible is a config-management tool that applies the same changes across a whole fleet). The empty --result-id "" tells it to use the single set of results in the file, so you get fixes for what actually failed on this host, not the entire benchmark.

# Remediation of rule 'xccdf_org.ssgproject.content_rule_sshd_disable_root_login'
if [ -e "/etc/ssh/sshd_config" ]; then
    LC_ALL=C sed -i "/^\s*PermitRootLogin\s\+/Id" "/etc/ssh/sshd_config"
fi
printf '%s\n' "PermitRootLogin no" >> "/etc/ssh/sshd_config"
 
# Remediation of rule 'xccdf_org.ssgproject.content_rule_service_firewalld_enabled'
SYSTEMCTL_EXEC='/usr/bin/systemctl'
"$SYSTEMCTL_EXEC" unmask 'firewalld.service'
"$SYSTEMCTL_EXEC" start  'firewalld.service'
"$SYSTEMCTL_EXEC" enable 'firewalld.service'

Read the script, then run it, then scan again. That last step is the one people skip, and it is the only proof the fix took. Watch the exit code while you are at it: oscap returns 0 when every rule passes, 2 when the scan ran fine but some rules failed, and 1 for a real error like a missing datastream. In automation, exit code 2 is the normal case and must not abort the run, which is why the wrapper script further down catches it on purpose.

Turn one scan into a trend

A host is hardened on the day you scan it, and it starts drifting the next morning. Someone opens a port for a demo. A package upgrade ships a new default. An on-call engineer sets PermitRootLogin yes at 3 a.m. to clear an outage and never sets it back. This is also what an intruder does once they land on a box: they turn root SSH login back on for easy re-entry, stop firewalld so their tools can call out, disable the audit daemon so their steps are not recorded. Every one of those is a CIS control. A scan you run on a schedule turns each of them into a specific failed rule and a lower score, which means your compliance baseline doubles as tampering detection. Wire it up with a small wrapper script and a systemd timer. Think of the timer as an alarm clock for the machine: it wakes at a set time, runs one job, and goes back to sleep. (systemd, the thing that provides these timers, is the service manager that starts and supervises background programs on modern Linux.)

#!/usr/bin/env bash
set -euo pipefail
 
DS=/usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml
PROFILE=xccdf_org.ssgproject.content_profile_cis_server_l1
OUT=/var/log/oscap
STAMP=$(date +%F)
mkdir -p "$OUT"
 
# oscap exits 2 when some rules fail (expected), 1 on a real error.
rc=0
oscap xccdf eval \
  --profile "$PROFILE" \
  --results "$OUT/results-$STAMP.xml" \
  --report  "$OUT/report-$STAMP.html" \
  "$DS" || rc=$?
if [ "$rc" -eq 1 ]; then
  logger -t oscap-scan "scan ERROR (exit 1)"; exit 1
fi
 
score=$(xmllint --xpath 'string(//*[local-name()="score"])' "$OUT/results-$STAMP.xml")
logger -t oscap-scan "CIS L1 score $STAMP: $score"

[Unit]
Description=Weekly CIS Level 1 scan (OpenSCAP)
Wants=network-online.target
After=network-online.target
 
[Service]
Type=oneshot
Nice=19
IOSchedulingClass=idle
ExecStart=/usr/local/sbin/oscap-scan.sh

[Unit]
Description=Run the CIS scan every Monday morning
 
[Timer]
OnCalendar=Mon *-*-* 03:00:00
Persistent=true
RandomizedDelaySec=30m
 
[Install]
WantedBy=timers.target

Now the score line in the system journal becomes a time series: 96, 96, 96, then one Monday it reads 88 with two new failures logged beside it. That is your signal. You open that week's report.html, see exactly which controls flipped from pass to fail, and go find out who changed them and whether it was allowed. The report is a to-do list for hardening, and also the fastest way to answer "what changed on this host, and when" after the fact.

One more thing about that score. A perfect CIS pass means the host matches a known configuration standard, and that is all it means. It says nothing about an unpatched application, a weak network boundary, a leaked credential, or an attacker already logged in. Benchmarks check configuration, not the code you deploy or the traffic you allow. Treat CIS as the floor every host has to clear and as a tripwire for drift, then layer patching, least privilege, monitoring, and response on top.

Two habits make this pay off. Commit each dated results file and report into a git repository, so you keep a timestamped record of every host's state for the next audit and a clean diff whenever something moves. And send the score line your wrapper logs to wherever your other alerts land, so a drop from 96 to 88 pages a human the morning it happens, not at the next quarterly review.

Try this

Work through “Turn one scan into a trend” yourself on a sandbox you can throw away, following the commands above in order. Then break one step deliberately and re-run, so you have seen the failure before it finds you.

Takeaway

The trap worth remembering here: remediation can lock you out of your own box. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related