The host threat model
What an attacker does after landing.
The hardening course was about locks. Better doors, fewer keys, thicker walls. This course starts from an uncomfortable assumption: someone got past all of it and is standing in your kitchen right now. The questions change. Not "how do I keep them out" but "what will they do next, how would I see it, and what do I do about it." That shift is the whole difference between advanced Linux security and configuration work. You stop staring at the walls and start thinking like the person already inside, because you cannot catch an intruder whose next ten moves you cannot predict.
An attacker who lands on a Linux host almost never arrives as root (the all-powerful administrator account that can read and change anything). They arrive as a nobody. A web application running as the service account www-data gets exploited, and now it runs their code. A low-privilege SSH (Secure Shell, the encrypted remote-login protocol) key gets copied off someone's laptop. A poisoned software dependency runs inside whatever account pulled it in. The front door was a bug in your app. The person who walked through it has exactly the permissions your app had, which is usually very little.
Meet the intruder: you are www-data now
When an attacker gets code running on a compromised host, the first thing they check is who they are. So run the same command they run.
That output is the attacker's whole world for now. uid=33 (the user ID number) is www-data, a service account with no login shell, no real home directory, and no membership in the sudo or adm groups (the groups that would hand it administrative power or the right to read system logs). It cannot read /etc/shadow (the file that holds password hashes). It cannot write to /etc, /usr, or /lib. What it does have is code execution and time. A competent intruder does not spend that time smashing things. They spend it looking.
The first thing they do is look around
A burglar who is already inside does not start by kicking in doors. They walk room to room, trying handles, noticing which windows are unlatched, before they touch anything. On a host, that walk-through is called enumeration, and the single most valuable thing to try is a password-free path to root. Every attacker checks their own sudo (the tool that lets an approved user run commands as root) rights first.
Read that last line the way an attacker reads it. Someone on the ops team gave the web service a no-password shortcut to restart its own worker as root, because it was convenient during a deploy. To them it is a small quality-of-life fix. To an intruder it is a lead worth chasing, and to you it is a line item the next time you audit sudoers. This is the pattern for the rest of the lesson: the same fact is a tool for one side and a signal for the other.
The other classic enumeration step is hunting for SUID binaries. SUID is the set-user-ID bit, a flag on a program that says "run me with the permissions of the file's owner, not the person who launched me." A normal program runs as you. A SUID-root program is a key stamped "acts as root while it runs." Some programs genuinely need it. passwd has to write /etc/shadow, which only root may touch, so passwd is SUID root by design. The attacker lists every one of them, because any SUID-root binary that can be tricked into running their input becomes a road from www-data to root.
That is the default set on a stock Ubuntu 22.04 host. Notice pkexec on the list. In 2022 a bug in it, tracked as CVE-2021-4034 (a CVE is a Common Vulnerabilities and Exposures identifier, the industry's shared serial number for a specific flaw) and nicknamed PwnKit, turned that one SUID binary into an instant local root for any user on millions of machines. The binary was supposed to be there. The bug inside it was not. That is why the list itself is not the alert; a change to the list is.
A SUID-root binary is not automatically a bug. passwd, sudo, and mount all need the bit to do their jobs. The danger is a SUID-root file you did not expect: one dropped by an attacker after they already reached root, or one added quietly by a package update. So do not eyeball this list against memory. Snapshot it right after you build the host, store the snapshot somewhere the host itself cannot write to, and diff against it on a schedule. A single new line is the whole signal, and one new line hides easily among twenty.
Escalate, persist, act
Once someone has a foothold, they want three things in roughly this order. Escalate: turn www-data into root, because root can read every file and change every setting. Persist: make sure a reboot, a patch, or a killed process does not evict them, since a foothold you lose in a week is close to worthless. Act: do the thing they actually came for, which is copy your database, hop to the next host, or rent out your CPU (central processing unit, the chip that runs the work) to mine cryptocurrency for them. Each phase touches the host differently, and one of them is far easier to see than the others.
Persistence has to touch disk, and disk remembers
To be sure they can get back in, an intruder cannot rely on remembering where they hid. They have to leave a copy of the key somewhere the house will hand back to them on its own. On Linux that means a file the system reads at boot or on a schedule: a cron job (a task the time-based scheduler cron runs automatically at set times), an entry in an SSH authorized_keys file, or a systemd unit. systemd (the init system, the first user-space program the kernel starts, running as process number 1, which then launches everything else on the machine) is the favorite, because it can relaunch the attacker's program the instant it dies and it hides in plain sight among hundreds of legitimate units.
Here is what a planted unit looks like. A defender who did not know better would scroll straight past it.
[Unit]Description=Network Connectivity MonitorAfter=network-online.target[Service]Type=simpleExecStart=/usr/local/bin/.netmonRestart=alwaysRestartSec=30[Install]WantedBy=multi-user.target
Everything about that file is designed to be boring. The description sounds like ops housekeeping. Restart=always means if you kill the process, systemd brings it right back in thirty seconds. The program it runs is a hidden dotfile, .netmon, tucked in /usr/local/bin where a stray binary raises no eyebrows. After the attacker runs systemctl enable --now network-check, it starts on every boot for the life of the machine. So how do you find a file whose entire job is to look unremarkable? You do not read it. You notice when it appeared.
The -newermt flag filters to files changed after a date you name. On a stable production server, almost nothing in these directories moves between planned package updates. A .service file with a modification time of 03:12, on a night when no change window was scheduled, is your lead. The problem with this command is that it is a spot check. It only works if you remember to run it, and it only catches what is still on disk when you look. You want a tripwire, not a flashlight.
auditd turns the write into a record
auditd (the Linux Audit daemon, a logging service wired directly into the kernel) records events the moment they happen, whether or not anyone is watching the screen. Instead of asking "what changed since Tuesday," you ask the kernel to shout the instant anything touches a directory you care about. You do that with watch rules.
-w /etc/systemd/system/ -p wa -k systemd_persist-w /etc/cron.d/ -p wa -k cron_persist-w /root/.ssh/authorized_keys -p wa -k ssh_persist
Read one line. -w names a path to watch. -p wa fires on writes and on attribute changes such as permission edits. -k attaches a label so you can search for these events later. Drop the file in /etc/audit/rules.d/, load it with augenrules --load, and the kernel now reports every write into those locations. When the attacker copies their unit into place, the tripwire fires, and you can pull the event back by its label.
Read that from the bottom up. The SYSCALL line is the gold. auid=deploy is the audit login user ID, the original human identity recorded at login, and neither sudo nor su can change it. So even though the write ran as uid=root, you can see it traces back to the deploy account, which means a stolen or misused deploy credential is your real problem. comm=cp and exe=/usr/bin/cp tell you cp did the writing. The proctitle line shows the exact command, including the staging directory /tmp/.x the attacker copied from. You now know who, what, and when, in one event. The attacker could rename the unit, hide the binary, and pick the dullest description on earth. None of that helps them, because they cannot write into /etc/systemd/system without crossing a watch on that directory. That is what a detection that survives contact means: you key on the thing the attacker has to do, not the thing they get to choose.
Assume breach is a budget line, not a mood
Assuming a breach is coming is not pessimism, and it is not giving up on prevention. It is admitting that prevention has a failure rate above zero, so you spend part of your effort on the alarm and the runbook instead of pouring all of it into the lock. A host you hardened but cannot see into is a house with a beautiful deadbolt and no windows to look through: once someone is inside, they own the dark. Every detection in this course exists because it catches one specific move an attacker is forced to make. Writing detections like that means knowing those moves cold, which is why the next lessons take privilege escalation and persistence apart in detail before you write a single rule. The engineer who understands the technique writes a rule that still fires when the attacker gets careful. The one who pasted a rule off a blog catches only the ones who were not really trying.
Before you move on, prove your own tripwire is real. Load the persistence rules above, reboot the host, and check what is actually live.
If your three lines come back after the reboot, the tripwire is persistent and an attacker writing into those paths will trip it. If it comes back empty, the rules only loaded at runtime and vanished when the machine restarted, which means a patient attacker would have nothing to set off. Making that ruleset survive a reboot is the first concrete thing to fix, and it is where the next lesson begins.
Try this
Work through “Assume breach is a budget line, not a mood” yourself on a sandbox you can throw away, following the commands above in order. Then break one step deliberately and re-run, so you have seen the failure before it finds you.
Takeaway
The trap worth remembering here: a tripwire an attacker can cut is not a tripwire. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.