Skip to content

Add openscap plugin for CIS benchmark reporting - #863

Draft
ianpittwood wants to merge 2 commits into
mainfrom
feat/openscap-plugin
Draft

ianpittwood wants to merge 2 commits into
mainfrom
feat/openscap-plugin

Conversation

@ianpittwood

Copy link
Copy Markdown
Contributor

Closes #88

Adds an openscap plugin to Bakery that reports CIS benchmark compliance for Bakery-managed images. It only reports and never modifies images, so humans or agents can turn the findings into Containerfile changes. The design and the spike evidence behind it are in posit-bakery/docs/openscap-plugin-spec.md.

What's included

  • Commands: bakery openscap scan, bakery openscap fetch (warms the content cache) and bakery openscap profiles <product>....
  • How it scans: oscap runs as root in a pinned scanner container that Bakery builds locally. The target image is mounted read-only, using --mount type=image on Docker 28+ or an exported temporary volume on older engines. Only Docker is needed on the host.
  • Content: pinned ComplianceAsCode release (v0.1.82), checked against its sha512 checksum and cached. The datastream is picked from the image's own /etc/os-release, so the mapping works beyond Ubuntu.
  • Built-in container tailoring: removes host- and kernel-only rule families (sysctl, kernel modules, partitions, bootloader, audit, AIDE, time sync).
  • Validation fails closed. Before results are trusted, the scanned OS must match the image, the content's installed-OS check must pass, a canary rule must evaluate, and at least one rule must pass or fail. Missing evidence, an unreadable image, a failed export or a timeout is an ERROR, never a pass.
  • Output: a console table plus summary.json and summary.md per image in results/openscap/<image>/. Each failing rule lists its CIS section, rationale and a bash fix, and the summary records the image ID, scanner and oscap versions, content and tailoring used. The HTML report, raw XCCDF/ARF results and the fix script are available on request.
  • bakery.yaml options: profile, contentVersion, datastream, tailoring, skipRules, failOn (default none) and formats. Each has a matching CLI flag.
  • Also: a setup-openscap action (caches and fetches content), Renovate pins for the content version, and updated docs.

Results

Image Pass Fail N/A Status
Ubuntu 22.04 (basic test image) 127 7 189 OK
Ubuntu 24.04 130 8 194 OK
Debian 12 129 7 191 OK
Ubuntu 26.04 — — — SKIPPED: no CIS profile upstream yet
Alpine — — — SKIPPED: unsupported OS

The Ubuntu 24.04 row is with the default gate. The same scan with --fail-on medium reported BREACH. The Debian row was scanned before the stricter validation commit and has not been re-run since.

Testing

  • 77 unit tests, with Docker and HTTP mocked.
  • test_integration.py is opt-in with BAKERY_TEST_OPENSCAP=1 because it downloads about 180 MB of CIS-titled content, which we keep out of public CI. It builds a weakened image (telnet installed, chmod 777 /etc/passwd) and checks that exactly those rules flip to fail, under both scan strategies.

Follow-ups / open items

  • Have a person review the host-only rule list (HOST_ONLY_RULE_PATTERN in content.py).
  • root_path_all_dirs and root_path_no_dot likely fail falsely when scanning offline. Candidates for skipRules.
  • Confirm that Renovate's extractVersion strips the v from the ComplianceAsCode tag.
  • Not tested on arm64 or on Docker engines older than 28.
  • Optionally publish the scanner image to GHCR instead of building it locally.

Adds `bakery openscap scan|fetch|profiles`. Scans run oscap in a locally built
scanner container with the target image mounted read-only (docker image mount,
or an exported volume on older engines), using pinned and sha512-verified
ComplianceAsCode content selected from the image's /etc/os-release. A built-in
container tailoring drops host/kernel-only rules; each scan is validated
against the target OS before results are trusted. Outputs a table plus
JSON/markdown summaries with per-rule CIS refs and bash remediation.

Also adds setup-openscap, Renovate pins, docs and the design spec.
Address review findings:
- error (not skip) when the image is missing or os-release can't be read
- validation fails when ARF, installed-OS result or canary rule is absent
- check docker export exit status; partial rootfs is an error
- name scanner containers and force-remove them on timeout/interrupt;
  reclaim root-owned output before temp-dir cleanup
- publish cached content only after checksum/extraction completes
- resolve bakery.yaml datastream/tailoring paths against the context
- pin scanner base digest and openscap-scanner version
- record image ref/ID, scanner, oscap version and tailoring in summary.json
@github-actions

Copy link
Copy Markdown

Test Results

2 554 tests  +79   2 552 ✅ +77   10m 14s ⏱️ + 1m 25s
    1 suites ± 0       2 💤 + 2 
    1 files   ± 0       0 ❌ ± 0 

Results for commit f595ef6. ± Comparison against base commit 012b5ba.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement openscap tool in bakery for image hardening

1 participant