Skip to content

Applying CIS Benchmarks on Linux Without Breaking Production

A safe rollout plan for CIS Benchmarks on RHEL and Ubuntu: scan with OpenSCAP or USG, know which controls break servers, tailor exceptions, and remediate in stages.

●By 12 min read

Sooner or later an auditor, a customer questionnaire or a cyber-insurance form asks whether your Linux servers follow the CIS Benchmarks. The fastest answer is to run a remediation script for the whole profile and re-scan until it reports green. That is also the fastest way to find out that your container hosts no longer route traffic, your automation account is locked out, and a full audit log has put a database server into single-user mode.

This guide is about applying the benchmark deliberately: scan first, learn which controls are disruptive, record exceptions with a reason, and roll the rest out in stages. For general hardening without a compliance framework, start with our Linux server hardening guide; this one assumes the framework is the requirement.

What a CIS Benchmark Actually Contains

Each benchmark is a versioned document for one distribution release, such as Ubuntu 24.04 or RHEL 9, with a few hundred numbered recommendations. Every recommendation belongs to a profile and is marked Automated (a tool can check it) or Manual (a human has to judge it):

ProfileIntentTypical impact
Level 1 - ServerBaseline that should not stop a normal server workingLow, but not zero
Level 2 - ServerDefence in depth on top of Level 1Can remove functionality on purpose
Level 1 - WorkstationBaseline for desktopsKeeps the GUI stack
Level 2 - WorkstationStrict desktop profileHigh
ToolDistributionsNotes
OpenSCAP + SCAP Security GuideRHEL, Rocky, Alma, Fedora, SUSEFree; ships CIS-aligned profiles and generates Ansible or bash fixes
Ubuntu Security Guide (usg)Ubuntu LTSRequires an Ubuntu Pro subscription (free for limited personal use)
CIS-CAT ProMost distributionsCIS's own assessor, for CIS SecureSuite members
Ansible Lockdown rolesRHEL family, Ubuntu, othersCommunity roles per benchmark; every control can be toggled

Match the version β€” a benchmark for Ubuntu 22.04 numbers and words things differently from the 24.04 one. Pin the benchmark version in your exception register so findings stay comparable.

Level 1 is the realistic target β€” for most fleets, Level 1 Server plus a short list of Level 2 controls you actually want is defensible. Full Level 2 is designed to cost functionality.

Aligned is not certified β€” community profiles track CIS closely, but when an auditor specifically requires CIS-CAT results, use CIS-CAT for the evidence and the free tools for day-to-day work.

Scan First, Change Nothing

The first run is pure measurement. It tells you how far off you are and, more usefully, which failures are cheap and which ones need a decision:

# RHEL / Rocky / Alma

$ sudo dnf install -y openscap-scanner scap-security-guide

$ oscap info /usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml | grep 'Id:.*cis'

Id: xccdf_org.ssgproject.content_profile_cis

Id: xccdf_org.ssgproject.content_profile_cis_server_l1

Id: xccdf_org.ssgproject.content_profile_cis_workstation_l1

Id: xccdf_org.ssgproject.content_profile_cis_workstation_l2

$ sudo oscap xccdf eval \

--profile xccdf_org.ssgproject.content_profile_cis_server_l1 \

--results /root/cis-results.xml \

--report /root/cis-report.html \

/usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml \

| awk '/^Rule/ {r=$2} /^Result/ && $2=="fail" {print r}' > /root/cis-failed.txt

$ wc -l < /root/cis-failed.txt

# Ubuntu LTS with Ubuntu Pro attached

$ sudo pro enable usg

$ sudo apt install -y usg

$ sudo usg audit cis_level1_server # HTML and XML results under /var/lib/usg/

The plain cis profile is Level 2 β€” in the SCAP Security Guide, content_profile_cis is Level 2 Server. Most people want cis_server_l1 first.

Exit codes β€” oscap xccdf eval returns 0 when every rule passes, 2 when at least one fails, and 1 on an error. CI jobs that treat any non-zero code as a crash need to know the difference.

Scan a representative set β€” a web node, a database, a container host and a bastion fail differently. Five scans by role tell you more than fifty identical ones.

Read the HTML report β€” each failed rule links its rationale and the exact remediation. That is the material for your exception decisions.

The Controls That Break Servers

Most recommendations are harmless: banners, file permissions on /etc/shadow, disabling unused filesystems, logging settings. A minority cause nearly all the outages. Review these before any automated remediation runs:

Control areaWhat breaksSafer approach
noexec on /tmp and /dev/shmInstallers, build tools and Java libraries that unpack and run native code from /tmpApply, then set TMPDIR for the few apps that need it; test package installs
Separate /var, /var/log, /var/log/audit, /homeCannot be done on a running server without repartitioningMark as an exception now; fix it in the image or kickstart
IP forwarding disabledDocker, Kubernetes, VPN gateways and routers stop passing trafficException for container hosts and gateways; enforce elsewhere
squashfs kernel module disabledSnap packages on Ubuntu no longer mountException wherever snaps are in use
Host firewall with default denySSH and application ports, plus conflicts with Docker's own rulesWrite the allow rules first and apply from a console session
SSH: no root login, AllowGroups, strict ciphersAutomation that logs in as root; old clients and network gearMove automation to a sudo user first; test old clients
pam_faillock lockoutMonitoring or automation with a stale password locks the accountKey-only automation accounts; know faillock --user NAME --reset
Password expiry on existing accountsService accounts with passwords expire months laterCheck with chage -l; lock service account passwords instead
auditd disk-full actionsadmin_space_left_action set to single or halt takes the host downDedicated audit partition plus disk alerts, before enabling
Immutable audit rules (-e 2)Rule changes need a rebootFinalise rules on canaries before turning it on
cron.allow and at.allowUsers not listed can no longer use crontabList every service account that owns a crontab
Default umask 027Deployed files unreadable by the web server groupSet UMask= per systemd service where needed
GRUB passwordConsole recovery needs the passwordStore it in your secrets vault before rollout

Firewall and Docker β€” Docker manages its own iptables or nftables rules, and published container ports can bypass a host firewall you thought was default-deny. Read our Docker guide for Linux admins before enforcing firewall controls on container hosts, and generate rule sets with the firewall rules tool.

SSH changes β€” the SSH section overlaps heavily with our SSH security best practices. Run sshd -t after every change and keep a second session open until a fresh login succeeds.

Cloud images β€” provider agents sometimes depend on things the benchmark disables. Azure's provisioning, for example, uses UDF-formatted media, so disabling the udf module needs testing on Azure VMs.

sponsored

Tailoring: Exceptions With a Paper Trail

An exception is not a failure if it is documented. Auditors generally accept a deviation that has a reason, a compensating control and an owner. Keep that register in version control next to the tailoring file that implements it:

# cis-exceptions.yaml (your own format; one entry per deselected rule)

- rule: xccdf_org.ssgproject.content_rule_sysctl_net_ipv4_ip_forward

scope: role=container-host

reason: Docker and Kubernetes networking require IPv4 forwarding

compensating: nftables forward chain restricted to bridge interfaces

owner: platform-team

review: 2027-04-01

- rule: xccdf_org.ssgproject.content_rule_kernel_module_squashfs_disabled

scope: os=ubuntu

reason: snapd mounts every snap as a squashfs image

owner: platform-team

review: 2027-04-01

# OpenSCAP: build a tailoring file that deselects the rules

$ autotailor --output tailoring.xml \

--unselect xccdf_org.ssgproject.content_rule_sysctl_net_ipv4_ip_forward \

/usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml \

xccdf_org.ssgproject.content_profile_cis_server_l1

$ oscap info tailoring.xml # shows the new tailored profile ID

$ sudo oscap xccdf eval --tailoring-file tailoring.xml \

--profile <tailored-profile-id> \

--report /root/cis-tailored.html \

/usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml

# Ubuntu USG: generate a tailoring file, edit it, audit with it

$ sudo usg generate-tailoring cis_level1_server tailor.xml

# set selected="false" on the rules you are excepting, then:

$ sudo usg audit --tailoring-file tailor.xml

GUI option β€” SCAP Workbench (scap-workbench) opens the data stream, lets you tick rules off, and saves a tailoring file. Useful the first time, when you are still reading what each rule does.

Tailor by role, not by host β€” one tailoring file per server role keeps exceptions reviewable. Per-host exceptions turn into an unmaintainable spreadsheet within a year.

Values are tailorable too β€” many rules have variables such as password length or the audit log size. Changing a value is often better than deselecting the rule entirely.

Remediate in Rings

Never run a full remediation directly on production. Generate the fixes as code, read them, and roll them out the same way you would roll out any other risky change:

# Pre-generated playbooks ship with scap-security-guide

$ ls /usr/share/scap-security-guide/ansible/ | grep cis

rhel9-playbook-cis.yml

rhel9-playbook-cis_server_l1.yml

# ...

# Or generate one that honours your tailoring file

$ oscap xccdf generate fix --fix-type ansible \

--tailoring-file tailoring.xml --profile <tailored-profile-id> \

--output cis-l1-tailored.yml \

/usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml

# Ring 0: a lab clone of the production image

$ ansible-playbook -i inventory cis-l1-tailored.yml --limit lab --check --diff

$ ansible-playbook -i inventory cis-l1-tailored.yml --limit lab

# Ring 1: a few non-critical production hosts, with console access ready

$ ansible-playbook -i inventory cis-l1-tailored.yml --limit canary

Use the task tags β€” the generated tasks are tagged with the rule name and with complexity, disruption and reboot hints such as low_disruption and reboot_required. Applying the low-disruption set first gets most of the score with little risk.

--check is not proof β€” check mode shows intended changes, but some tasks cannot predict their effect without running. The lab ring exists because of that gap.

Define healthy before you start β€” SSH from the bastion, application health checks, backups, monitoring agents and scheduled jobs. Check all of them after each ring, and after the next reboot, since several controls only take effect at boot.

On Ubuntu β€” sudo usg fix --tailoring-file tailor.xml applies the tailored profile. Run it on a snapshot first, exactly like the Ansible path.

Keep a way back β€” a VM snapshot, an image you can redeploy, or at minimum an out-of-band console. A firewall or SSH control gone wrong locks out the tool that would roll it back.

Staying Compliant: Images and Drift

Remediating existing servers is the expensive path. The cheap path is to build new ones compliant, then check that they stay that way:

# RHEL kickstart: apply the profile during installation

%addon com_redhat_oscap

content-type = scap-security-guide

profile = xccdf_org.ssgproject.content_profile_cis_server_l1

%end

# Weekly drift scan from a systemd timer or cron job

$ sudo oscap xccdf eval --tailoring-file /etc/cis/tailoring.xml \

--profile <tailored-profile-id> \

--results /var/log/cis/results-$(date +%F).xml \

/usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml

Partition at build time β€” the separate /tmp, /var, /var/log and /var/log/audit controls are trivial in a kickstart or Packer template and painful anywhere else.

Scan results are evidence β€” keep the XML results with a date. Auditors usually want proof of continuous compliance, not one green report.

Review exceptions on schedule β€” the review date in the register matters. A reason that was true for an old kernel or Docker version may no longer hold.

Schedule it properly β€” build the weekly scan with our systemd timer generator or the cron generator, and pull the remaining items from the server hardening checklist.

Frequently Asked Questions

Should I apply CIS Level 1 or Level 2 to my Linux servers?

Start with Level 1 Server. It is designed to be practical on a working server, and it covers most of what auditors look for. Level 2 is defence in depth that deliberately removes functionality; adopt individual Level 2 controls where they fit your risk rather than the whole profile.

Is OpenSCAP the same as an official CIS-CAT scan?

No. The SCAP Security Guide profiles are aligned with the CIS Benchmarks and are a good free tool for scanning and remediation, but CIS-CAT Pro is CIS's own assessor. If an auditor specifically requires CIS-CAT output, produce that for the evidence and use OpenSCAP or USG for daily work.

Which CIS controls are most likely to break a server?

The usual ones are disabling IP forwarding on container hosts, enforcing a default-deny firewall, strict SSH settings that block root automation or old clients, pam_faillock lockouts, password expiry on service accounts, auditd disk-full actions that halt the system, noexec on /tmp, and disabling squashfs on Ubuntu systems that use snaps.

Can I be CIS compliant with exceptions?

Yes, in practice. Auditors generally accept documented deviations that state the reason, a compensating control and an owner, and that are reviewed periodically. Implement them as a tailoring file so the scan reflects the agreed baseline rather than reporting the same known failures every time.

How do I apply CIS hardening to Ubuntu?

Attach Ubuntu Pro, enable and install the Ubuntu Security Guide with pro enable usg and apt install usg, and run usg audit cis_level1_server to measure. Generate a tailoring file for exceptions, test usg fix with that tailoring file on a snapshot or lab machine, then roll it out in stages.

sponsored

Related Tools

See also