seccomp: filter syscalls

RuntimeDefault and a curated custom profile.

Every time a container reads a file, opens a socket, or starts a process, it has to ask the host kernel to do the actual work. That request is a syscall (system call), the one doorway between a running program and the kernel. A modern x86-64 Linux kernel offers around 350 of them. A typical web service uses maybe 40 to 70. The other three hundred do nothing for you and quite a lot for an attacker, because kernel bugs tend to live right at that doorway. Dirty Pipe (CVE-2022-0847) is the case everyone remembers. It took splice and write, two ordinary calls the default profile happily allows, and turned them into a way to overwrite files marked read-only. Researchers demoed it as a container escape. You can't block those two without breaking normal file reads and writes, which is exactly the argument for taking away the ones you can spare. A syscall you never expose is a syscall nobody can turn against the kernel. Taking them away is a job for a bouncer with a guest list. Name on the list, you walk through. Name missing, you get stopped at the door and never reach the bar. That bouncer is seccomp (secure computing mode), the kernel feature that checks every syscall a process makes against an allow-list. Calls on the list go through to the kernel. Everything else is refused and handed back an error, before the kernel does a scrap of the work.

You don't write that guest list yourself. Every container Docker starts on the default runtime already carries one, a profile named RuntimeDefault. It permits the calls ordinary programs make and refuses the rest. Docker's docs name about 44 of the refused ones, and they are the dangerous or barely-used calls: mount and pivot_root for rearranging filesystems, the kernel-module loaders init_module and finit_module, the keyring calls keyctl, add_key and request_key, plus reboot and swapon. You get all of that switched on for free. The catch is how easily it comes back off, usually by accident, usually because somebody was debugging an incident and never put it back. So the first move is always the same. Check that a filter is actually loaded.

Check the filter is on, then watch it bite

# the seccomp mode lives in every process's status file.
# 2 means a BPF (Berkeley Packet Filter) program is loaded and enforcing.
$ docker run --rm alpine grep -E 'Seccomp' /proc/self/status
Seccomp:	2
Seccomp_filters:	1
# now watch RuntimeDefault deny a real syscall. the kernel keyring is a single
# host-wide store with no namespace of its own, so the profile blocks add_key.
$ docker run --rm alpine sh -c 'apk add -q keyutils; keyctl add user demo secret @s'
add_key: Operation not permitted

That EPERM ("Operation not permitted", the kernel's generic refusal) came from the bouncer, not from a missing capability. The two failures look identical on screen, which is why people misdiagnose them. To work out which control said no, take one of them away and try the call again. Drop the profile and it sails straight through. You get a second thing out of that test for free: you see what an unconfined container looks like from the outside, so you know how to hunt for one.

# seccomp=unconfined removes the filter entirely. the keyring call now succeeds.
$ docker run --rm --security-opt seccomp=unconfined alpine \
    sh -c 'apk add -q keyutils; keyctl add user demo secret @s'
723451890                       # the new key's serial. no filter, no bouncer.
# so how do you find an unconfined container across a running fleet? ask the daemon.
$ docker inspect --format 'seccomp={{.HostConfig.SecurityOpt}} priv={{.HostConfig.Privileged}}' web
seccomp=[seccomp=unconfined] priv=false
# a healthy container reads:  seccomp=[] priv=false   (an empty list means the built-in default)

Build a profile that fits one app

The default is a floor, not a ceiling. It has to stay loose enough that almost any program still runs, so it keeps allowing hundreds of calls your particular service will never make. A per-app profile turns the logic around. Deny everything, then allow back only what this one binary touches. You don't guess that list. You measure it. Trace the process while you exercise every route it serves, or run it under an audit profile (SCMP_ACT_LOG, which writes down each syscall instead of blocking it), then read off the set that actually showed up.

# trace the app through a full exercise of its routes, then read the summary.
$ strace -f -c ./api --serve          # Ctrl-C once every code path has run
strace: Process 12 attached
^Cstrace: Process 12 detached
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 28.9    0.006129          15       420           epoll_pwait
 21.3    0.004512          10       466           futex
 11.8    0.002501           5       472         6 read
  9.1    0.001940           5       418           write
  6.0    0.001270           4       311         4 openat
  5.2    0.001100           4       305           close
  5.1    0.001080           8       130           mmap
  4.4    0.000930           4       208           fstat
  3.1    0.000660           7        96           rt_sigprocmask
  2.6    0.000550           9        61           accept4
  1.6    0.000340           8        44           mprotect
  0.9    0.000190           5        41           brk
------ ----------- ----------- --------- --------- ----------------
100.00   0.021202                  2972        10 total
# these are the busy syscalls; strace -c lists every one the process made.
# the full, de-duplicated last column is your allow-list. nothing else goes in.

{
  "defaultAction": "SCMP_ACT_ERRNO",
  "defaultErrnoRet": 1,
  "archMap": [
    {
      "architecture": "SCMP_ARCH_X86_64",
      "subArchitectures": ["SCMP_ARCH_X86", "SCMP_ARCH_X32"]
    }
  ],
  "syscalls": [
    {
      "names": [
        "read", "write", "openat", "close", "fstat", "newfstatat",
        "statx", "lseek", "mmap", "mprotect", "munmap", "brk",
        "rt_sigaction", "rt_sigprocmask", "ioctl", "execve",
        "arch_prctl", "set_tid_address", "futex", "epoll_pwait",
        "accept4", "getuid", "getgid", "exit_group"
      ],
      "action": "SCMP_ACT_ALLOW"
    }
  ]
}

A from-scratch allow-list is the tightest option and the easiest one to get wrong. Miss a single syscall the runtime needs during startup and the container dies in a way that tells you nothing useful. Swap defaultAction from SCMP_ACT_ERRNO to SCMP_ACT_KILL_PROCESS and the debugging gets harder still. The kernel fires SIGSYS and kills the process where it stands, with no error return for strace to catch, only a container that is suddenly gone. The gentler first move, and the one Docker's own docs walk you through, is to copy the default profile and delete only the calls you have proof you don't need. Below, that's the chmod family. Changing a file's mode gets refused, and everything else the app leans on carries on working.

# no-chmod.json is Docker's default profile with chmod/fchmod/fchmodat removed.
$ docker run --rm --security-opt seccomp=./no-chmod.json alpine \
    chmod 400 /etc/hostname
chmod: /etc/hostname: Operation not permitted     # fchmodat is no longer on the list
# confirm the profile a running container actually carries (a fleet-wide audit query):
$ docker inspect --format '{{ .HostConfig.SecurityOpt }}' api
[seccomp=/etc/docker/seccomp/no-chmod.json]

So the whole control comes down to one list sitting between your process and the kernel. RuntimeDefault handles the obscure calls for you. A custom profile goes tighter, because you know exactly which binary is running. Matching happens on the syscall number, and on the arguments too if you write argument rules. Get either one wrong and you have picked one of two bad outcomes: an app that breaks, or a hole you cannot see.

When an app misbehaves under a tight profile, go and read the audit log, write down which syscall got refused, and widen the profile by that one name. Reaching for seccomp=unconfined does make the symptom disappear. It also throws away every other rule you wrote.

Unconfined seccomp in production is a smell. Treat it the way you treat --privileged: temporary, attached to a ticket, and with an expiry date somebody actually checks.

After a change window, run the same check you ran before it. Confirm the status file still reads Seccomp: 2, paste the command and its output into the ticket, and refuse to sign the change off if the reading moved. Controls rarely get removed on purpose. They get removed at 2am and nobody writes it down.

Pick the tightest scope the workload will tolerate, and stop there. On one host that habit is worth very little. Across a fleet it is worth a great deal, because every image, every pipeline and every rebuild inherits whatever you settled on the first time.

Try this

Start a container under the default seccomp profile and call a syscall the profile blocks, reboot or clock_settime for example, through a tiny program or through docker run flags. Then use docker inspect to confirm the filter really is on. Here is a short run you can paste straight in.

$ docker run --rm alpine sh -c 'wget -qO- https://example.com >/dev/null && echo net-ok'
net-ok
$ docker inspect --format='{{.HostConfig.SecurityOpt}}' $(docker run -d alpine sleep 30)
[]
$ docker run --rm --security-opt seccomp=unconfined alpine echo unconfined-demo
unconfined-demo
$ CID=$(docker run -d alpine sleep 30); docker inspect -f '{{.HostConfig.MaskedPaths}}' $CID | head -c 80; echo; docker rm -f $CID >/dev/null
[/proc/asound /proc/acpi /proc/kcore ...]

Takeaway

Leave RuntimeDefault switched on everywhere. Write a tighter profile for the handful of services that would hurt most if somebody got into them. And never let seccomp=unconfined settle in as a quiet default in someone's compose file. Filtering syscalls is how you shrink the slice of kernel your container is allowed to touch at all.

Related