CoursesLinux hardeningMinimize services & exposure

Minimize services & exposure

Every listener you remove is risk removed.

Intermediate10 min · lesson 8 of 16

A server with eight listening services is a house with eight doors onto the street. It does not matter that you only ever use one of them. Every door is a lock that can be picked, a hinge that can rust, a frame that might have been fitted wrong at the factory. The safest door is the one that was never installed. That is the whole point of minimizing services. Code you do not run cannot be exploited, and a port you do not open cannot be reached. So hardening a Linux box starts by asking a blunt question of every running program: does this machine actually need you?

Take an honest inventory

Before you remove anything, you need to know what is there. The fastest way is ss, short for socket statistics, a tool that reads the kernel's live list of network sockets. The kernel is the core of the operating system, the part that owns the hardware and the network. Ask ss for the TCP listeners and the program behind each one. TCP (Transmission Control Protocol) is the connection-based traffic that most services speak.

~/secopslog — bash
$ sudo ss -tlnp
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process LISTEN 0 4096 127.0.0.53%lo:53 0.0.0.0:* users:(("systemd-resolve",pid=642,fd=14)) LISTEN 0 128 0.0.0.0:22 0.0.0.0:* users:(("sshd",pid=901,fd=3)) LISTEN 0 244 0.0.0.0:5432 0.0.0.0:* users:(("postgres",pid=1123,fd=7)) LISTEN 0 128 0.0.0.0:631 0.0.0.0:* users:(("cupsd",pid=1201,fd=7)) LISTEN 0 128 [::]:22 [::]:* users:(("sshd",pid=901,fd=4)) LISTEN 0 128 [::]:631 [::]:* users:(("cupsd",pid=1201,fd=9))

Four flags do the work. -t limits it to TCP, -l to sockets in the LISTEN state (open and waiting), -n keeps ports as numbers instead of translating 22 into ssh, and -p shows the owning process, which is why you need sudo. The column that matters most is Local Address:Port. 0.0.0.0 means every network interface the machine has, reachable from outside. 127.0.0.1 (and 127.0.0.53) is the loopback address, reachable only from the machine itself. [::] is the same idea as 0.0.0.0 for IPv6 (the newer, longer addressing scheme that is slowly replacing the classic four-number addresses). Read this list and two things jump out. PostgreSQL, a database, is on 0.0.0.0:5432, and a print server is on 0.0.0.0:631. Both are answering the whole network. A database and a printer, facing the internet.

This is close to what an attacker sees. Point a port scanner like nmap (network mapper, the standard tool for probing which ports a host has open) at the box from outside and they get the same open ports, minus the process names, because they do not have your sudo. Every name on your list is a program they can start fingerprinting for a known bug. One habit worth keeping: add -u to also list UDP (User Datagram Protocol, the connectionless kind that DNS lookups and service discovery use), since some listeners never show up in a TCP-only view.

Match each listener to a running service

ss tells you what is on the network. systemd tells you what is running and meant to be. systemd is the init system, the very first process to start at boot (process ID 1, the root of the whole process tree) and the parent that supervises everything after it. Anything it manages is called a unit. Ask it which services are running right now.

~/secopslog — bash
$ systemctl list-units --type=service --state=running
UNIT LOAD ACTIVE SUB DESCRIPTION avahi-daemon.service loaded active running Avahi mDNS/DNS-SD Stack cron.service loaded active running Regular background program processing daemon cups.service loaded active running CUPS Scheduler dbus.service loaded active running D-Bus System Message Bus [email protected] loaded active running PostgreSQL Cluster 14-main ssh.service loaded active running OpenBSD Secure Shell server systemd-journald.service loaded active running Journal Service systemd-logind.service loaded active running User Login Management systemd-resolved.service loaded active running Network Name Resolution LOAD = Reflects whether the unit definition was properly loaded. ACTIVE = The high-level unit activation state, i.e. generalization of SUB. SUB = The low-level unit activation state, values depend on unit type. 9 loaded units listed. Pass --all to see loaded but inactive units, too.

Now pair the two lists. cups.service is running and owns port 631. CUPS (the Common Unix Printing System) turns a machine into a print server. Do you print from this box? Almost never. avahi-daemon is running too. Avahi advertises services on the local network, the magic that lets laptops find printers and shared folders on their own. A headless server (one with no monitor, keyboard, or desk user, just a network cable) has no reason to shout its presence to the LAN (local area network). Two clear candidates for removal, and neither is subtle once you look.

Stop, disable, mask, purge

There are four strengths of turning something off, and the difference matters. Stopping a service closes the door now, but it swings open again at the next reboot. Disabling it removes the boot-time startup link so it stays shut across reboots, though you can still start it by hand. Adding --now does both at once. Masking is stronger: it points the unit at /dev/null (the system's black hole, a destination that swallows anything sent to it), welding the door shut so nothing can start it, not even another service that depends on it. Purging is the most thorough: it removes the package and its config from disk entirely, taking the door out of the wall. Start with disable for the two you found.

~/secopslog — bash
$ sudo systemctl disable --now avahi-daemon sudo systemctl disable --now cups
Synchronizing state of avahi-daemon.service with SysV service script with /usr/lib/systemd/systemd-sysv-install. Executing: /usr/lib/systemd/systemd-sysv-install disable avahi-daemon Removed "/etc/systemd/system/dbus-org.freedesktop.Avahi.service". Removed "/etc/systemd/system/multi-user.target.wants/avahi-daemon.service". Removed "/etc/systemd/system/sockets.target.wants/avahi-daemon.socket". Synchronizing state of cups.service with SysV service script with /usr/lib/systemd/systemd-sysv-install. Executing: /usr/lib/systemd/systemd-sysv-install disable cups Removed "/etc/systemd/system/multi-user.target.wants/cups.path". Removed "/etc/systemd/system/printer.target.wants/cups.service". Removed "/etc/systemd/system/sockets.target.wants/cups.socket".

Read the output. Disabling deletes the symlinks that told systemd to start these at boot. Look at how much came off for cups. Three links, not one: cups.path, cups.service, and cups.socket. They travel together because the service file lists the other two under its Also directive, so a single disable unlinks all three. That looks complete. It is not. Disabling only removes those boot-time links, and --now stopped exactly one unit, the service. The socket unit is still running this very second. That gap is the trap in the next section.

The socket-activation trap

systemd has a trick called socket activation. Think of a receptionist who does not keep every specialist sitting at a desk all day. The receptionist holds the phone line open, and only when a call comes in do they page the specialist, who arrives just in time. Here systemd is the receptionist. It holds the port open, and it starts the real program only when the first connection lands. That saves memory. It also means a port can be open while the service you think owns it is stopped. You stopped cups a moment ago. Look at 631 again.

~/secopslog — bash
$ sudo ss -tlnp | grep ':631'
LISTEN 0 128 0.0.0.0:631 0.0.0.0:* users:(("systemd",pid=1,fd=30)) LISTEN 0 128 [::]:631 [::]:* users:(("systemd",pid=1,fd=32))

The port is still listening, and the owner is now systemd, pid=1, not cupsd. The service is stopped, but systemd is holding the doorway, ready to relaunch CUPS the instant something knocks. Here is the catch. disable --now stopped the one unit you named, cups.service. It never touched cups.socket, and a disabled unit that is already running keeps running until you stop it. The socket is a separate unit, still active, still holding 631. Stop the socket, then mask both so nothing can bring them back.

~/secopslog — bash
$ sudo systemctl stop cups.socket sudo systemctl mask cups.service cups.socket sudo ss -tlnp | grep ':631'
Created symlink /etc/systemd/system/cups.service → /dev/null. Created symlink /etc/systemd/system/cups.socket → /dev/null. (the final ss prints nothing: port 631 is gone)

The lesson generalizes. Whenever you stop a service and the port stays open under systemd, pid=1, you are looking at socket activation, and the unit still holding the door ends in .socket. Stopping the .service never closes it. Stop and mask that .socket to shut the port for good. CUPS works this way, and so do a number of systemd's own helpers; the tell is always systemd, pid=1, owning the listener.

Stopping a service can leave its socket wide open
Socket-activated units split the listener (name.socket) from the worker (name.service). Turn off only the service and systemd keeps the socket open, ready to relaunch the worker on the first packet, so your port scan still lights up. Stop and mask the .socket too, then re-run ss to confirm the port is really gone. If ss shows systemd, pid=1 owning a listener, that is socket activation, not a bug.

Bind to the house, not the street

Some services you cannot remove, but only the machine itself needs to reach them: a database that backs one application, a metrics endpoint scraped by a local agent. Picture an intercom wired only to rooms inside the house versus one wired to a panel out on the street. Same device, wildly different exposure. The wiring here is the listen address. PostgreSQL was sitting on 0.0.0.0:5432, open to the whole network. If only the local app talks to it, bind it to loopback instead.

/etc/postgresql/14/main/postgresql.conf
# Was: listen_addresses = '*' (every interface, reachable from the network)
listen_addresses = 'localhost' # loopback only; unreachable from off the box

Change the setting, restart PostgreSQL, then verify. The listen address is read only when the server starts, so a plain reload will not move it, and a config edit you did not check is a hope, not a control.

~/secopslog — bash
$ sudo systemctl restart postgresql@14-main sudo ss -tlnp | grep ':5432'
LISTEN 0 244 127.0.0.1:5432 0.0.0.0:* users:(("postgres",pid=1487,fd=7)) LISTEN 0 244 [::1]:5432 [::]:* users:(("postgres",pid=1487,fd=8))

Loopback now, 127.0.0.1 and its IPv6 twin ::1, not 0.0.0.0. Even if the firewall in front of this box were misconfigured tomorrow, the database still would not answer a remote connection, because it is not listening on any address the network can route to. A local bind and a default-deny firewall are two independent reasons the service is unreachable from outside, and a hardened host wants both.

Take away the attacker's tools

Listeners are only half the job. Think about what an intruder finds lying around once they are inside. Someone who lands a shell goes looking for tools to build a payload, scan the internal network, and move sideways: a compiler like gcc (the GNU Compiler Collection), an interpreter like Python, a network utility like nc (netcat, a Swiss-army knife for opening raw connections), and legacy remote-login clients like telnet and rsh (remote shell). Using what is already installed instead of downloading anything is called living off the land, and it works because nothing new shows up for a detector to catch. A web server that never compiles code does not need gcc sitting in an open shed.

~/secopslog — bash
$ sudo apt purge -y telnet telnetd rsh-client rsh-server ftp
Reading package lists... Done Building dependency tree... Done Reading state information... Done The following packages will be REMOVED: ftp* rsh-client* rsh-server* telnet* telnetd* 0 upgraded, 0 newly installed, 5 to remove and 0 not upgraded. After this operation, 612 kB disk space will be freed. (Reading database ... 74213 files and directories currently installed.) Removing telnetd (0.17-44) ... Removing telnet (0.17-44) ... Removing rsh-server (0.17-25build1) ... Removing rsh-client (0.17-25build1) ... Removing ftp (0.17-36) ... Processing triggers for man-db (2.10.2-1) ...

You will not strip a host to bare metal, and you should not try. Removing a compiler can break packages that build a kernel module at install time. The rule is narrower: take away what a server of this role has no business running, and be able to say why every tool that stays is there. Legacy clients like telnet, rsh, and ftp send passwords across the network in the clear anyway, so they are almost always safe to remove and worth removing twice over. This is the bare-metal version of the distroless idea from the container course, where an image ships with no shell and no package manager. The less that is present, the less there is to attack and to abuse.

Baseline it, then watch for drift

The last move turns a one-time cleanup into a standing control. Once a host is minimal and understood, write down the ports it is supposed to have. A night guard who has memorized which cars belong in the lot spots the strange one at 3am immediately. Capture the known-good set once, then compare the live set against it on a schedule.

~/secopslog — bash
$ # capture the known-good baseline once, right after hardening ss -tlnH | awk '{print $4}' | sort -u | sudo tee /etc/baseline-ports.txt >/dev/null # later, on a timer, compare the live listeners against the baseline ss -tlnH | awk '{print $4}' | sort -u | comm -13 /etc/baseline-ports.txt -
0.0.0.0:6379

Here -H drops the header row, awk pulls the Local Address:Port column, and comm -13 prints only the lines that are new (present in the live list, absent from the baseline). The write needs root, which is why the capture pipes into sudo tee rather than a plain redirect; the shell that runs > cannot write to /etc, only the command on the left of the pipe runs under sudo. Empty output means nothing changed. A line means a port appeared. That one is a Redis (an in-memory data store, often used as a cache) instance someone spun up for a test and left running on every interface with no password. On a supposedly known-good server you will turn these up more often than you would like: a leftover from a package's install script, a debug listener, a monitoring agent nobody documented. Baseline the expected ports per host role, alert on anything new, and treat an unexplained open port as something to close or justify, never to accept. What is listening is, exactly, what can be attacked.

Triage for a listening port
You found a listening port. What do you do with it?
Serves external clients
Keep it, bind on purpose
0.0.0.0 plus an explicit firewall allow-rule; every public port justified
Only used on the box
Bind to 127.0.0.1
loopback only; the network cannot route to it
Nothing needs it
disable --now, then mask or purge
handle the .service and its .socket unit, then re-check with ss
You can't explain it
Treat it as a finding
investigate, then close it or write down why it stays
Quick check
01You run systemctl disable --now cups, but ss -tlnp still shows port 631 listening, now owned by systemd, pid=1. Why?
Correct — --now stopped the service, not the socket. Stop and mask cups.socket, then confirm with ss.
Incorrect — No. disable --now takes effect at once; the separate, still-running .socket unit is the real reason.
Incorrect — No. ss reads the kernel's live socket table on every run; nothing is cached.
Incorrect — No. A firewall filters traffic; it never opens a listening socket.
02You've already run systemctl disable on a service, but you're worried another unit that depends on it could still pull it back to life. What does masking the unit do that disabling does not?
Incorrect — that is purging; masking leaves the package installed but unstartable.
Incorrect — disable --now also stops a running service; masking's distinction is blocking future starts, not the stop itself.
Incorrect — disable already handles boot-time autostart; masking additionally blocks manual and dependency-triggered starts.
Correct — a masked unit is welded shut, while disable only removes the boot-time autostart link and can still be started by hand or by a dependency.
03You edit postgresql.conf to set listen_addresses = 'localhost', run systemctl reload postgresql, then check with ss, but PostgreSQL is still listening on 0.0.0.0:5432. Why, and what fixes it?
Correct — the listen address is a startup-time setting, so a reload re-reads runtime options but won't change the bound socket.
Incorrect — 'localhost' is accepted and binds both 127.0.0.1 and ::1; the real problem is that reload doesn't apply a listen-address change.
Incorrect — ss reads the kernel's live socket table on every run and caches nothing, so the socket really is still on 0.0.0.0.
Incorrect — a firewall filters traffic, it never decides which address a service binds to; the bind address is chosen by the app at startup.

Try this

Work through “Baseline it, then watch for drift” yourself on a sandbox you can throw away, following the commands above in order. Then break one step deliberately and re-run, so you have seen the failure before it finds you.

Takeaway

The trap worth remembering here: stopping a service can leave its socket wide open. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related