Scan Your PC Against a Benchmark With OpenSCAP

OpenSCAP ships as a command called oscap. On Ubuntu or Debian:

$ sudo apt-get install -y openscap-scanner

On Fedora, RHEL, Rocky or AlmaLinux it is one package plus the benchmark content in another:

$ sudo dnf install -y openscap-scanner scap-security-guide

Confirm it is there and note the version — you will want it if you ask for help later:

$ oscap --version

You should see a line like OpenSCAP command line tool (oscap) 1.3.9 and a block of supported specifications. oscap is the engine; it does nothing on its own until you hand it a benchmark to check against, which is the next step.

Why this matters. OpenSCAP is the reference implementation of SCAP, the set of standards the US government and most auditors use to describe a security configuration in a machine-readable way. The same scanner reads content written for CIS Benchmarks and for DISA STIGs — you learn one tool, not one per standard.

A benchmark arrives as one big XML file called a datastream. The content package drops them here — list what you have:

$ ls /usr/share/xml/scap/ssg/content/

Ask the scanner what is inside the one that matches your OS. oscap info lists the profiles — the named rule sets you can run:

$ oscap info /usr/share/xml/scap/ssg/content/ssg-ubuntu2404-ds.xml

You will see profiles such as cis_level1_server, cis_level1_workstation and stig. Three words worth pinning down now: a benchmark is the whole standard, a profile is one named subset of it (CIS Level 1 is the “sensible defaults” tier; Level 2 is stricter and breaks more things), and a rule is a single check inside a profile, like “password maximum age is 365 days or fewer”.

The gotcha. If the only Ubuntu datastream your package installed is older than your release — say you are on 24.04 but the file is ssg-ubuntu2204-ds.xml — the scanner’s applicability check will decide the benchmark does not apply to your machine and mark every single rule “not applicable”. The fix is to grab the matching release straight from the project:

$ curl -fsSL -o ssg.zip https://github.com/ComplianceAsCode/content/releases/download/v0.1.77/scap-security-guide-0.1.77.zip

$ unzip -q ssg.zip && cd scap-security-guide-0.1.77

Now you have ssg-ubuntu2404-ds.xml in the current folder, and the rest of the commands can use that short name. (Check the releases page for the newest version number.)

“Not applicable” on all 400-odd rules does not mean your machine is clean. It means the scanner looked at the benchmark, looked at your OS, and decided they were strangers. A score of zero problems from a scan that checked nothing is the most dangerous kind of green. — Nyx

Why this matters. This is the number-one reason a first SCAP scan looks suspiciously perfect. Matching the content version to the OS version is half the skill; the tool will not warn you, it will just quietly pass on everything.

Run the CIS Level 1 Server profile against your machine. The two output flags are the point — one machine-readable results file, one human-readable HTML report:

$ sudo oscap xccdf eval \
    --profile xccdf_org.ssgproject.content_profile_cis_level1_server \
    --results cis-results.xml \
    --report cis-report.html \
    ssg-ubuntu2404-ds.xml

It scrolls a line per rule — Title, Rule, Result — and finishes in a minute or two. On the fresh Lab 00-style VM this was written against, the tally came out at 147 pass, 30 fail, 217 not applicable, with a handful not checked. Your numbers will differ; that is the whole idea.

One thing that surprises people: the command exits with code 2, not 0. That is not an error — oscap returns 2 to mean “the scan ran fine and at least one rule failed”. A script checking for success should treat 0 and 2 as “it ran” and only 1 as “it broke”.

Why this matters. Notice what did not happen: nothing on your machine changed. xccdf eval is a read-only audit. It is safe to run on a production box at any time, which is exactly why it is the first half of every compliance process — you can measure before you ever touch a setting.

Open cis-report.html in a browser. (On a VM with no desktop, copy it to your host, or open it over your file share.) The top shows a compliance score and a bar; below it, every rule as a coloured row you can click to expand.

Four results you will see, and what each one means:

• pass — the setting already matches the benchmark. Nothing to do.
• fail — the setting does not match. This is your to-do list.
• notapplicable — the rule does not apply to this system (a rule about a package you do not have, or a partition you did not create). Not a problem, but worth skimming so a real gap does not hide in the pile.
• notchecked / unknown / error — the scanner could not test it automatically, usually because the check needs a human to look. These do not count for or against you; they are a manual review list.

Click any failing rule and expand it. You get a plain-English title, a Rationale explaining why the setting matters, an identifier tying it to the CIS or STIG reference, and often the exact remediation. That expandable rule is the unit everything else is built from — the next step reads one on its own.

The score at the top is seductive and almost meaningless on its own. “72% compliant” sounds like a grade; it is really “72% of the rules we happened to check”. Read the failures, not the percentage. — Nyx

Why this matters. This HTML report is the thing you actually hand to an auditor or a manager. It is self-contained, it cites the standard for every rule, and it is dated. Learning to read it is learning to read the output of every commercial compliance scanner, because most of them are a prettier front end over this same engine.

Pick a small, safe rule to understand fully. Set Password Maximum Age is a good one. You can run a single rule by id — much faster than the whole profile when you are iterating:

$ sudo oscap xccdf eval \
    --profile xccdf_org.ssgproject.content_profile_cis_level1_server \
    --rule xccdf_org.ssgproject.content_rule_accounts_maximum_age_login_defs \
    ssg-ubuntu2404-ds.xml

On a stock machine this returns:

$ Result    fail

What is it actually checking? This rule reads /etc/login.defs and wants PASS_MAX_DAYS to be 365 or fewer. A fresh Ubuntu ships it as 99999 — effectively “passwords never expire”. Confirm the current value yourself:

$ grep PASS_MAX_DAYS /etc/login.defs

A benchmark rule is just three things wearing a long identifier: a place to look, a value to want, and a reason. Once you see that, the scary XML is only a list of “check here for this”. — Nyx

Why this matters. A rule you cannot explain is a rule you cannot defend fixing — or safely decide to skip. Half of real compliance work is reading a failing rule, understanding what it wants and why, and then making a deliberate call. The scanner finds the rule; the judgement is yours.

Change the one setting the rule wants. This edits a real system file, so you are glad now you snapshotted the VM. Set the maximum age to 365:

$ sudo sed -i 's/^PASS_MAX_DAYS.*/PASS_MAX_DAYS\t365/' /etc/login.defs

Now run the exact same single-rule scan from step 5 again. The only thing that changed is one line in one file:

$ Result    pass

That flip from fail to pass is the whole loop in miniature: the scanner measured, you changed exactly what it asked for, the scanner re-measured and agreed. Nothing was guessed.

Nova will tell you the point was the green word. The point was that you can now prove the change worked, instead of saying “I edited the file, trust me”. Proof beats trust in every audit ever held. — Nova

Why this matters. “Re-scan to confirm” is the step people skip, and it is the one that catches the typo, the wrong file, the setting that needs a service restart. In a real environment you fix a batch of rules, then re-run the whole profile and watch the fail count drop. Same loop, bigger numbers.

You fixed one rule by hand to understand it. For many rules, OpenSCAP can write the fix script for you. Generate a bash remediation for the whole profile — but do not run it yet:

$ oscap xccdf generate fix \
    --profile xccdf_org.ssgproject.content_profile_cis_level1_server \
    --fix-type bash \
    ssg-ubuntu2404-ds.xml > remediation.sh

Open remediation.sh and read it. It is thousands of lines — one block per failing rule, each labelled with the rule it fixes. This is the dividing line of the whole field: assessment (the scan in step 3) only measures and is always safe; remediation (this script) changes the machine and is not. A hardening script can disable a service you needed, tighten a mount option that breaks an app, or enforce a password policy that locks out a service account. Auto-remediation is a loaded tool: read it, apply it on a snapshot, and re-scan — never pipe it straight into a shell on a box that matters.

To feel the loop one more time by hand, fix the default umask too. The CIS rule wants UMASK set to 027 in the same file (stock Ubuntu ships 022):

$ sudo sed -i 's/^UMASK.*/UMASK\t027/' /etc/login.defs

Re-run the full profile scan from step 3. Your fail count is now lower than it was, and the report you hand over tells the story: what you measured, what you changed, and the green to prove it.

Why this matters. Every GRC platform, every “one-click hardening” tool and every CIS-hardened cloud image is this split underneath: assess, then remediate, then re-assess. Knowing which half you are running — the safe one or the dangerous one — is the difference between an audit and an outage.