Keys, MFA & bastions

Strong auth and a single front door.

A password is a secret you say out loud. Say it at the right door and you're in. The trouble is that saying it is the whole trick: anyone who overhears it, phishes it, or guesses it can say the same word and walk in behind you. An SSH key (Secure Shell, the encrypted protocol you use to log into remote machines) works more like a physical key that never leaves your pocket. You prove you hold it without ever handing it over. This lesson is about making that proof strong, adding a second lock to the one door that's left, and funneling every login through a single guarded entrance you can watch.

How A Key Proves You're You

A key pair is two matching files. The private key is the physical key in your pocket. The public key is the shape of the lock that only that key opens. You hand out the lock shape freely and keep the key. When you connect, the server sends your client a random challenge. Your client signs that challenge with the private key and sends back the signature. The server checks the signature against the public key it already holds. If it matches, you're in, and the private key itself never crossed the wire. Overhearing the whole conversation gives an attacker nothing they can replay.

Generate a modern pair. The ed25519 algorithm (a current public-key signature scheme) is the right default: the keys are tiny, signing is fast, and the security is strong. Give it a passphrase. The passphrase encrypts the private key file on disk, so a key file copied off your laptop is a locked box, not a working key.

That produced two files. The private key lives at ~/.ssh/id_ed25519 and never leaves this machine. The public key, the .pub file, is the one you copy to servers. The ssh-copy-id helper appends it to the remote account's ~/.ssh/authorized_keys, which is the list of public keys allowed to log in as that user.

Notice the permissions: 600, owner read and write only. This matters. The SSH server refuses to trust an authorized_keys file (or a .ssh directory) that other users can write to, because a writable list of allowed keys is a writable list of allowed people. That refusal is the StrictModes check, and it's on by default. If key login ever fails for no obvious reason, wrong permissions are the first thing to check.

Two habits keep the private key private. Load it into ssh-agent once at the start of your session (the agent holds the decrypted key in memory, so you type the passphrase a single time and the key never gets scattered into scripts or config files). And the day someone leaves the team, delete their public-key line from every authorized_keys. A key you forget to remove is a door you forgot to lock. A private key never belongs in a git repository or on a server; the moment it lands in either place, treat it as leaked and generate a fresh one.

Turn Off What Attackers Actually Hammer

Here's the part people forget. Installing a key does not remove the password door. As long as password login is switched on, every server on port 22 still accepts guesses, and the public internet is full of bots doing nothing but guessing, all day, forever. You close that door in the server's config (sshd is the background process, or daemon, that answers SSH connections).

# only keys get in; no passwords, no root logins
PasswordAuthentication no
PubkeyAuthentication yes
PermitRootLogin no

Modern SSH reads drop-in files from /etc/ssh/sshd_config.d/, so you add a small file of your own instead of editing the big shared one. Test the config before you apply it, then reload. Running sshd -t parses the config and prints nothing if it is valid, which is the Unix way of saying 'fine.' A reload re-reads the config without dropping the connections that are already open.

Now look at what the box was taking before. Even with passwords off, the bots keep knocking, and every knock is a log line. This is the defender's view: a steady drizzle of invalid usernames from addresses all over the world.

Each 'Invalid user' line is a bot trying a common account name, getting nowhere, and moving on. With password login off and no such user, there is nothing to brute force, so the noise is harmless. What you actually watch for is the shape that isn't a bot: repeated tries against a real username, from one address, over a long stretch. That is someone who did their homework, and it deserves a closer look and maybe a firewall block.

A Second Lock On The Only Door Left

With passwords gone, the key is the only way in, which makes the key worth protecting twice. It works like a bank card. The card in your wallet plus the PIN in your head, and a thief needs both to take your money. MFA (multi-factor authentication) is that same idea applied to a login. It asks for two different kinds of proof: something you have, like the key or a hardware token you hold, plus something you know, like a PIN or a short rotating code. One of the two on its own gets nobody in. There are two common ways to add it to SSH.

The first is a hardware security key, a small token like a YubiKey that speaks the FIDO2 and U2F standards (open standards for hardware-backed login). You generate a special key type, ed25519-sk (the 'sk' means security key), where the real secret lives inside the token and cannot be copied out. Logging in then requires the physical token to be plugged in and tapped. A key file stolen off your laptop is now useless, because the actual secret never sat in a file to begin with.

The verify-required flag adds a PIN check on top of the tap, so a stolen token still needs something you know. The second approach is TOTP (time-based one-time password, the rotating six-digit code an app shows you). Linux wires it in through PAM (Pluggable Authentication Modules, the framework that lets you stack extra login checks). You install the module (apt install libpam-google-authenticator), run google-authenticator to enroll a user and scan the QR code, add one line to the PAM stack, and then tell the SSH server to demand both a key and a code.

# ask for the rotating code as part of the SSH login
auth required pam_google_authenticator.so

# a valid key AND a keyboard-interactive step (the TOTP code via PAM)
AuthenticationMethods publickey,keyboard-interactive
KbdInteractiveAuthentication yes
UsePAM yes

One Front Door

Exposing SSH on every server to the internet is like giving every room in an office its own street entrance. That is a lot of doors to lock, watch, and patch. A bastion (also called a jump host) is the single staffed reception desk: the only machine reachable from outside, hardened and heavily logged, and the only thing your internal servers will accept SSH from. The attack surface drops from 'every host on port 22' to 'one gateway you watch closely.'

You do not want to log into the bastion and then type a second ssh command from there, because that puts your key, or an agent connection, on the machine most exposed to attack. ProxyJump avoids that. It opens an encrypted tunnel through the bastion and runs the real login end to end between your laptop and the internal host. The bastion passes bytes through and never sees the key you use for that internal host.

Host bastion
    HostName bastion.acme.internal
    User deploy
    IdentityFile ~/.ssh/id_ed25519
 
Host app-*
    HostName %h.acme.internal
    User deploy
    ProxyJump bastion         # ssh app-01 transparently hops through the bastion

Now ssh app-01 quietly hops through the bastion and lands on the internal host. One command, and you never typed the bastion's name.

On the internal hosts, close the loop two ways. Point the firewall so port 22 answers only the bastion's address, and pin the key itself to that source with a from= restriction inside authorized_keys. The from= clause turns a stolen public-key line into something that only works when the connection actually originates from the bastion.

from="10.0.5.8" ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIH8s0m2pQ7v9kZ2xR4tW6yB1cLmN3dF5gH7jK8sP0aQe deploy@laptop

Because every session now passes through one machine, the bastion is also your ledger. A single host's logs hold who connected, when, and where they went next. That is the operational payoff hiding inside the security one: a single place to audit access instead of scraping fifty scattered log files.

Before you call it done, prove the password door is actually shut. Force SSH to offer only a password and watch it bounce.

That 'Permission denied (publickey)' is the line you want to see. It means the server never even entertained the password path, and the only way in is the key you are holding.

Try this

Work through “One Front Door” yourself on a sandbox you can throw away, following the commands above in order. Then break one step deliberately and re-run, so you have seen the failure before it finds you.

Takeaway

The trap worth remembering here: keep one session open while you change sshd. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related