SSH hardening

Keys only, no root, minimal exposure.

SSH (Secure Shell, the encrypted protocol you use to open a command line on a remote machine) is the front door to almost every Linux server. It listens on port 22, the numbered doorway that incoming network connections arrive on, and that door faces the whole internet. That is exactly why it is the most attacked service online. On any server with a public address, automated programs ring the doorbell thousands of times a day, trying username and password pairs one after another, all day, forever. You do not have to take that on faith. You can watch it happen.

Read the attack before you stop it

On a modern Linux machine, every login attempt gets written to the system log. journalctl (the command that reads systemd's log, where systemd is the manager that starts and supervises background programs on most current Linux distributions) lets you page through it. Ask it for the SSH service over the last hour and filter down to the failures.

Every one of those lines is a bot, a hijacked machine somewhere running an attack script, guessing a password against accounts it hopes exist: root, admin, postgres, ubuntu. They are not being clever. They are trying volume. The way to shut down this entire category of attack is not a longer or stranger password. It is taking passwords out of the login path completely, so there is nothing left to guess.

Why keys beat passwords

A password is a code punched into a keypad. Given enough attempts a keypad code can be guessed, and these bots have nothing but time. A cryptographic key works on a different principle. Say you have a lock you can bolt onto a door, and one oddly shaped piece of metal that opens it. You can make thousands of copies of the lock and bolt them onto servers all over the world at no risk, because the lock by itself opens nothing. Only the single piece of metal works, and it never leaves your pocket. That is public-key authentication. The lock is your public key, which you hand out freely. The metal is your private key, which you guard. Guessing a valid private key by trial is arithmetically hopeless. An ed25519 key (a modern, fast key type built on elliptic-curve math) has so many possible values that trying them one by one would outlast the machine you are attacking by an unimaginable margin.

You make a key pair once, on your own laptop, not on the server. The private half stays with you. The public half is safe to copy anywhere.

For the server to accept that key, its public half has to be listed there, in a file called authorized_keys inside the target user's ~/.ssh folder. ssh-copy-id does that for you. It logs in once the old way, appends your public key to that file, and leaves. From then on your key alone gets you in.

One rule of order matters more than anything else in this lesson. Get key login working first, and only then turn passwords off. If you disable passwords before your key is in place, you will lock yourself out of the very server you are trying to secure. Log in with the key in a fresh terminal, confirm it works, then keep going.

The hardening file

sshd (the SSH daemon, the always-running background program that answers incoming connections) reads its settings from /etc/ssh/sshd_config. On Debian 12 and Ubuntu 22.04, that file opens with a line that pulls in every .conf file from a folder beside it called sshd_config.d. One design choice makes this easy to live with. Think of a stack of sticky notes on the door, and a guard who reads them from the top down. For almost every setting, sshd takes the first value it meets and ignores every later one. The include line sits near the top of the file, so anything you drop into sshd_config.d gets read before the shipped defaults further down. Your settings win, and you never touch the original file.

# Read first. This pulls your drop-in files in near the top of the config,
# and sshd honors the first value it sees for each setting.
Include /etc/ssh/sshd_config.d/*.conf
 
# ... the packaged default options follow below ...

So you write one small file. Create /etc/ssh/sshd_config.d/10-hardening.conf with the settings that matter, and give it a number low enough to sort near the front. One habit to keep here: in this config, comments live on their own lines. sshd does not read a comment tacked onto the end of a setting, and a stray one after a value can make the whole file fail its syntax check.

# Keys only. The single biggest change.
PasswordAuthentication no
PubkeyAuthentication yes
 
# Close the keyboard-interactive password path too, so no
# password prompt can slip back in through PAM.
KbdInteractiveAuthentication no
 
# Nobody logs in straight to root.
PermitRootLogin no
 
# Only members of this group may connect at all.
AllowGroups ssh-users
 
# Trim the attack surface and tighten the limits.
X11Forwarding no
MaxAuthTries 3
LoginGraceTime 20

No root at the door, and a guest list

Two lines in that file carry most of the weight. PermitRootLogin no means nobody connects straight into root, the all-powerful administrator account. Instead everyone logs in as a named human, then reaches for admin rights one action at a time with sudo (superuser do, which runs a single command with elevated rights and records who ran it). The master key that opens every room still exists. You do not let anyone walk in the front door already holding it. People come in as themselves, and they check the master key out at the desk, where the desk writes down their name. That written record is your audit trail. A shared root login destroys it, because every action shows up as 'root' and you can never tell who did what.

AllowGroups ssh-users adds a guest list at the door. Even a valid account with a valid key is refused unless it belongs to the ssh-users group. Stack these together and an attacker now needs three separate things at once: the actual private key of a real user, that user's membership in ssh-users, and then a second secret, the sudo password, before they reach root. Take away any one and they stop at the step before. You create the group and add your user like this.

Apply it without locking yourself out

Editing this config is one of the few changes on a server that can shut you out of it for good. So apply it like someone who has been burned before. Validate the file first with sshd -t, which checks the syntax and prints nothing at all if it is happy. Then reload, which tells the running sshd to re-read its config without dropping the connections it already has, so your current session stays alive.

Validation catches typos, but it does not tell you what sshd actually decided after reading every included file. For that, ask sshd to print its effective settings with sshd -T, which dumps the real values it will use, with the keyword names in lowercase. This is how you prove the drop-in file took effect instead of hoping it did.

Now look at the same log you started with. On the defender's side, a good change shows up as a shift in what the log records: your real user getting in by key, and the bots that used to guess passwords now bouncing off before authentication even begins.

The first two lines are you, in by key. The last two are bots hitting a wall. 'Connection closed ... [preauth]' means the connection was dropped before any authentication succeeded, and with passwords off there is no path for them past that point. The noise continues. The danger is gone.

One last habit worth keeping. Run sshd -T | grep passwordauthentication on every server you manage, not one. A fleet is only as hard as its softest host, and the single machine that missed the drop-in file, the one still answering passwords, is exactly the one a bot will find first.

Try this

Work through “Apply it without locking yourself out” yourself on a sandbox you can throw away, following the commands above in order. Then break one step deliberately and re-run, so you have seen the failure before it finds you.

Takeaway

The trap worth remembering here: keep your session open until the new one works. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related