CoursesAdvanced Linux securityKernel rootkits & detection

Kernel rootkits & detection

When you cannot trust the kernel.

Advanced14 min · lesson 8 of 17

Every tool you trust to tell you what a Linux box is doing asks the same office for its answers. ps wants the list of running programs. ls wants the files in a directory. ss wants the open network connections. None of them look at the hardware themselves. They all phone the kernel (the core part of the operating system that talks to the hardware and hands out its resources) and repeat back whatever it says. A kernel rootkit is a bribe paid to that office. Once it is in place the kernel keeps answering, everything still sounds normal, and every answer about the attacker is a lie.

You met a weaker version of this trick earlier with LD_PRELOAD (a setting that makes a program load an attacker's library first so its own functions get quietly replaced). That fools one program at a time, and only programs that are dynamically linked. A statically linked binary, or one you compile fresh, walks right past it. A kernel rootkit sits one level deeper. It changes the answer at the source, so every program gets the same lie, whether it is ps, your monitoring agent, or a tool you copied onto the box thirty seconds ago. That is why it is the hardest thing on this list to find from the machine it lives on, and why the hardening course kept hammering on who is allowed to load kernel code.

What a kernel rootkit actually does

Most kernel rootkits arrive as a loadable kernel module, or LKM (code you can plug into a running kernel without rebooting). Legitimate ones are everywhere. Your network card's driver is an LKM. A malicious one loads the same way and then rewrites how the kernel answers a few specific questions. The usual move is intercepting system calls (the requests every program makes to get real work done, like getdents64 to list a directory or openat to open a file). When ls asks the kernel to list /tmp, the rootkit lets the honest listing happen, then removes its own files from the result before handing it back. Ask for the process list, and the attacker's process is filtered out. Ask for open ports, and the backdoor's connection is scrubbed. The kernel does the work truthfully and then edits the receipt.

On an old kernel this was done by overwriting the syscall table, the kernel's phone directory of which function handles each system call. Modern kernels keep that table read-only, so today's rootkits reach it by flipping the processor's write-protect bit for a moment, or they skip the table entirely and hook the functions themselves using ftrace or kprobes, the kernel's own built-in tracing machinery. The result is the same either way. Something now runs between the honest kernel function and the answer that leaves the building.

First signs: a kernel that admits it is dirty

Start with the cheap checks, knowing every one of them can be beaten by a good rootkit, because you are asking the suspect about itself. The most honest signal the kernel gives is the taint flag. The kernel keeps a private note about anything that might make it behave in unsupported ways, and loading out-of-tree or unsigned module code sets specific bits in that note.

~/secopslog — bash
$ cat /proc/sys/kernel/tainted dmesg | grep -i taint
12288 [ 61.402314] example: loading out-of-tree module taints kernel. [ 61.402512] example: module verification failed: signature and/or required key missing - tainting kernel

That 12288 is not random. It is two bits added together: 4096 means an out-of-tree module was loaded (code that did not ship with this kernel), and 8192 means an unsigned module was loaded. A stock server running only signed, in-tree modules reads 0 here. Anything else is worth explaining before you go home. The taint flag will not name the attacker, but a non-zero value on a box you believed was clean is a thread to pull.

Next, look at what claims to be loaded.

~/secopslog — bash
$ lsmod | head
Module Size Used by example 16384 0 xt_conntrack 16384 1 nf_conntrack 172032 1 xt_conntrack overlay 151552 0 binfmt_misc 24576 1 nls_iso8859_1 16384 1

Why lsmod will not save you

Here is the catch that trips people up. lsmod is only a pretty-printer for /proc/modules, and both read the exact same linked list inside the kernel. A competent rootkit's first act is to unlink itself from that list. One line of code, and it vanishes from lsmod, from /proc/modules, and usually from /sys/module/ too. So a module you can see may be worth investigating, but a module you cannot see proves nothing. Comparing lsmod against /proc/modules and hoping for a mismatch is a myth. They drink from the same well. The whole reason detection is hard is that the thing you would reach for to find the rootkit is the thing the rootkit already owns.

Detection is a cross-view problem

So you stop asking the machine about itself and start checking its story against sources it does not control. Three views work, and each one gets stronger the further it sits from the compromised kernel.

The network view. The host swears there is no connection on port 4444. Your firewall logs, a network tap, or a flow sensor on a different machine see packets moving to and from that box on 4444 every night at 2 a.m. The kernel can lie to the programs sitting on top of it. It cannot un-send a packet that a switch already forwarded. A connection that exists on the wire but never appears in ss output is one of the loudest rootkit tells there is.

The disk view. Power the box down, attach its disk to a trusted machine, or boot it from a known-good rescue USB, and list the same directories the live host showed you. Files the running system hid from ls are sitting right there when a clean kernel reads the disk. The rootkit only works while its own kernel is running. Read the filesystem from outside and its edits to reality evaporate.

The memory view, which is the strongest. Capture the machine's RAM to a file and analyze it offline, where the rootkit's code is not running to interfere with you.

~/secopslog — bash
$ # capture RAM with LiME (Linux Memory Extractor) sudo insmod ./lime.ko "path=/mnt/evidence/mem.lime format=lime" # hunt for a module that unlinked itself from the module list vol -f /mnt/evidence/mem.lime linux.hidden_modules
Volatility 3 Framework 2.7.0 Offset Name Size 0xffffffffc0a2e000 reptile 49152

Volatility (a memory forensics tool) walks the kernel's structures inside that snapshot the way the kernel itself would, except nothing is alive in a file to filter the results. linux.hidden_modules carves module structures straight out of memory and lists any that are missing from the official module list, which is exactly where a self-hidden rootkit gets caught. Related plugins expose the other lies: linux.check_syscall flags system call handlers that point somewhere other than the real kernel function, and comparing linux.pslist (which walks the process list) against linux.psscan (which scans memory for process structures the list conveniently forgot) surfaces a hidden process. On a live box you can get a weaker version of that same cross-check with unhide, which brute-forces every possible process ID and compares what the kernel admits to against what actually answers.

~/secopslog — bash
$ sudo unhide brute
Unhide 20130526 Copyright © 2013 Yago Jesus & Patrick Gouin License GPLv3+ : GNU GPL version 3 or later http://www.unhide-forensics.info NOTE : This version of unhide is for systems using Linux >= 2.6 Used options: [*]Searching for Hidden processes through kill(..,0) scanning Found HIDDEN PID: 3847

Signature scanners like chkrootkit and rkhunter still have a place. They know the fingerprints of common kits and will flag a lazy attacker in seconds. They will also miss anything custom, and they run on the possibly-compromised host, so treat a clean result as weak evidence, not a clean bill of health.

Where the truth still lives
Ask the host
lsmod / ps / ss
all report clean
the kernel answers
and edits the attacker out
Network view
firewall / tap / flow log
sees the backdoor's traffic
off-box sensor
a sent packet can't be recalled
Disk view
rescue boot or mount
hidden files are visible
clean kernel reads disk
no filter is running
Memory view
LiME capture + Volatility
hidden_modules, check_syscall
analyzed offline
rootkit code is inert
The compromised kernel controls the first column. It does not control the other three.

Close the door before anyone walks through it

Finding a kernel rootkit is hard. Stopping one from loading is not, and it is the same module control you set up in the hardening course. Three locks, strongest first.

Turn on module signature enforcement so the kernel refuses to load any module that is not signed by a key it trusts. An attacker with full root can write a malicious .ko file to disk all day; if the kernel will not load unsigned code, that file is inert. Check two things: whether enforcement is on, and which kernel lockdown mode you are in (lockdown is a kernel-wide setting that blocks the paths root would otherwise use to reach kernel memory).

~/secopslog — bash
$ # is signature enforcement on? cat /sys/module/module/parameters/sig_enforce # which lockdown mode is the kernel in? cat /sys/kernel/security/lockdown
Y none [integrity] confidentiality

Y means the kernel rejects unsigned modules. The bracketed word is the active lockdown mode: integrity refuses unsigned modules and closes the side doors into kernel memory, and turning on Secure Boot sets this for you automatically. If that first line says N and lockdown reads [none], an unsigned rootkit loads with nothing standing in its way.

If a machine has finished booting and never needs another module, take the ability away entirely.

~/secopslog — bash
$ sudo sysctl -w kernel.modules_disabled=1 sudo modprobe dummy
kernel.modules_disabled = 1 modprobe: ERROR: could not insert 'dummy': Operation not permitted

That switch is one-way. Once it is 1, nothing loads another module until the next reboot, not even root, which is the entire point. Flip it only after everything the box needs is already in place, or you will lock out a driver you actually wanted. A small systemd unit ordered late in boot is the usual home for it.

/etc/systemd/system/disable-module-loading.service
[Unit]
Description=Disable kernel module loading after boot
After=multi-user.target
[Service]
Type=oneshot
ExecStart=/sbin/sysctl -w kernel.modules_disabled=1
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target

Whatever else you do, record every attempt to load or unload kernel code, so a load you did not authorize leaves a mark you can alert on. Drop a rule in the audit daemon (auditd, the Linux service that records security-relevant events) covering all three of the module syscalls.

/etc/audit/rules.d/modules.rules
-a always,exit -F arch=b64 -S init_module -S finit_module -S delete_module -k module_ops
~/secopslog — bash
$ sudo augenrules --load sudo ausearch -k module_ops -i | tail -8
---- time->Fri Jul 17 02:14:03 2026 type=SYSCALL msg=audit(07/17/2026 02:14:03.221:788) : arch=x86_64 syscall=finit_module success=yes exit=0 ppid=2145 pid=3390 auid=root uid=root gid=root euid=root comm=insmod exe=/usr/bin/kmod key=module_ops
A rootkitted kernel cannot be cleaned in place
If you have real evidence of a kernel rootkit, everything the host tells you is now suspect, including whether your removal worked. Do not try to disinfect it. Capture evidence first (a RAM image with LiME and a full disk image), because the memory is gone the moment you power off, then rebuild the host from known-good install media. After that, rotate every credential, key, and token the box could have touched, because a kernel-level attacker could read all of them. Trying to 'clean' a compromised kernel is how attackers ride straight through your incident response and keep their foothold.
Quick check
01A host's lsmod looks normal and its own ss shows nothing on port 4444, yet a flow sensor on a different machine records traffic to that box on 4444 every night. Which reading fits best?
Incorrect — The sensor sits outside the kernel you suspect, which is exactly what makes it harder to tamper with than the host's own answers.
Incorrect — Kernels do not grow stale module lists on their own, and a reboot leaves whatever loads the code from disk free to run again.
Correct — The kernel decides what ss and lsmod print, but the frames were on the network before it had any chance to edit the story.
Incorrect — A self-hiding module removes itself from the very list lsmod prints, so an empty result is the expected outcome either way.
02On a server you believed was clean, cat /proc/sys/kernel/tainted returns 12288. What have you actually learned?
Correct — Read the value as a bitmask: 4096 and 8192 add up to exactly 12288, so the number is two separate facts rather than one.
Incorrect — Nothing here is a counter. A machine loading only signed code that came with the kernel reports zero and keeps reporting zero.
Incorrect — The value does not drift with uptime. It moves only when particular kinds of code load, which is why a non-zero reading needs an answer.
Incorrect — Plenty of legitimate vendor drivers taint a kernel the same way, so this is a lead you have to explain, not a verdict you can act on.
03Memory forensics confirms a module that hid itself from the running kernel on a production host. What do you do?
Incorrect — You would be asking a kernel the attacker owns to remove the attacker's code, and the loader sitting on disk puts it straight back.
Incorrect — Something on disk put that module into the kernel in the first place, so a reboot only starts the same cycle over again.
Incorrect — The scanner runs on top of the kernel you already distrust and only recognises published kits, so a clean pass proves very little.
Correct — RAM vanishes the moment you cut power, so it goes first. After that you replace rather than repair, and assume every key was readable.

When you have set the locks, prove them in one place. cat /sys/module/module/parameters/sig_enforce should say Y, cat /proc/sys/kernel/tainted should say 0, and sudo modprobe anything on a finished box should come back Operation not permitted. Three answers, and the deepest hiding place on the machine is shut before anyone tries to use it.

Try this

Work through “Close the door before anyone walks through it” yourself on a sandbox you can throw away, following the commands above in order. Then break one step deliberately and re-run, so you have seen the failure before it finds you.

Takeaway

The trap worth remembering here: a rootkitted kernel cannot be cleaned in place. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related