User-namespace remapping

Map container root to an unprivileged host UID.

February 2019. A bug catalogued as CVE-2019-5736 (CVE is the public numbering scheme every disclosed vulnerability gets) turned a running container into a full takeover of the machine underneath it. A malicious image could overwrite the host's copy of runc, the small program Docker calls to actually start a container, from inside the container. The next time an admin typed docker exec, they ran the attacker's code as real root on the host. One default assumption made that possible: the user the kernel numbers 0 inside your container (UID 0, where UID means user identifier) is the same UID 0 that owns every file on the host. Root in the box is root on the box. User-namespace remapping deletes that assumption. The hosts that already had it switched on read the advisory, checked, and went back to work, because their containers had never been host root in the first place.

A namespace is a one-way mirror. The process inside sees its own room and takes that room for the whole house. The user namespace, userns for short, is the mirror for identity: who you are, as a number. Switch on userns-remap and the daemon gives every container its own identity room. A process in there is UID 0, calls itself root, and is treated as root by everything inside the room. On the other side of the glass the kernel keeps a translation table. Container 0 is host 100000. Container 1000 is host 101000. The block runs 65,536 identities wide. Type whoami inside and you get root. Look at that same process from the host and you get user 100000, who owns nothing, belongs to nothing, and can open nothing that was not handed over on purpose.

Watch the damage without it

Root-equals-root hurts most where a container reaches out and touches host files, and that place is a bind mount: a host directory passed straight into the container, no copy, no translation. Mount a host path into an ordinary non-remapped container and its root writes host files as host root, whatever the person who typed the docker run line had in mind. A compromised app in that container can schedule a cron job (a command Linux runs on a timer, as whichever user the job names), drop an SSH key (secure shell, the remote-login protocol) into somebody's account, or leave behind a SUID binary (set-user-ID: a file flagged so that anyone who runs it runs it as the file's owner, usually root). Here is that write with nothing standing in its way. Watch the owner column.

# no userns-remap: container root writes a HOST root-owned file
$ id -un                                   # on the host we're a normal deploy user, not root
deploy
$ docker run --rm -v /etc/cron.d:/mnt alpine \
    sh -c 'echo "* * * * * root id > /tmp/pwned" > /mnt/backdoor'
$ ls -n /etc/cron.d/backdoor               # -n prints numeric owner instead of a name
-rw-r--r-- 1 0 0 31 Jul 16 09:12 /etc/cron.d/backdoor
#          ^ owned by UID 0 = real host root; an unprivileged admin just planted a root cron job

Switch the mapping on

You set this once, in the config file of the daemon (the Docker background service that creates and supervises containers), then restart it. Docker creates a service account named dockremap and books a block of host UIDs and GIDs in its name (GID is group identifier, the same idea applied to groups). The booking is written into /etc/subuid and /etc/subgid, the system's record of which ranges of identities an account is allowed to claim for a namespace, rather like a building's book of which flats a tenant may sublet. The value default tells Docker to create and look after that account and its range for you. Once the daemon is back up, the translation table is not hidden: you can read it out of /proc, the directory where Linux exposes live kernel state as ordinary files.

# /etc/docker/daemon.json  ->  { "userns-remap": "default" }   then:  systemctl restart docker
$ cat /etc/subuid
dockremap:100000:65536             # host UIDs 100000-165535 are this daemon's to hand out
$ docker run --rm alpine cat /proc/self/uid_map
         0     100000      65536   # inside-id 0  ->  host-id 100000, for 65536 consecutive ids

One detail trips people up. Inside a mapped container, root still shows a full capability set (a capability, or cap, is one slice of root's power, chopped into roughly 40 pieces so a process can hold one without holding all of them). Those caps are genuine. They are also fenced inside this user namespace. Take CAP_DAC_OVERRIDE (DAC stands for discretionary access control, the everyday owner/group/other permission check on every file). It lets the process walk past permission checks on files owned by its own mapped range. It buys nothing against host-root files, which sit outside the namespace entirely. The power is real. The blast radius is a rented block of user numbers that owns nothing worth having.

Now prove container root is harmless

Run the same kind of write again. The container still believes it is root. The kernel stamps every file it creates with the mapped host UID, and root-owned host files stay out of reach. That ls -n on the bind mount is your receipt: container root wrote as host 100000.

# remap ON: writes land as the mapped UID, and root-owned host files are unreachable
$ sudo mkdir -p /srv/data && sudo chown 100000:100000 /srv/data
$ docker run --rm -v /srv/data:/mnt alpine sh -c 'echo hi > /mnt/newfile'
$ ls -n /srv/data/newfile
-rw-r--r-- 1 100000 100000 3 Jul 16 09:20 /srv/data/newfile
#          ^ container "root" wrote as host UID 100000, not 0
$ docker run --rm -v /etc/cron.d:/mnt alpine sh -c 'echo x > /mnt/backdoor2'
sh: can't create /mnt/backdoor2: Permission denied      # /etc/cron.d is root-owned; UID 100000 can't

Find who quietly opted out

Remapping is a daemon-wide switch, so the question that matters is not whether you turned it on last quarter. It is whether some workload has since turned it off for itself with --userns=host and now runs as host root next to everything you protected. Two commands cover both levels. docker info tells you what the daemon is doing. docker inspect finds the containers that walked out on it.

# 1) is the daemon actually remapping?  look for name=userns
$ docker info --format '{{.SecurityOptions}}'
[name=seccomp,profile=builtin name=cgroupns name=userns]
 
# 2) which running containers opted OUT to the host user namespace?
$ docker ps -q | xargs -r docker inspect \
    --format '{{.Name}} userns={{if .HostConfig.UsernsMode}}{{.HostConfig.UsernsMode}}{{else}}mapped{{end}}'
/web     userns=mapped
/legacy  userns=host        # <-- runs as REAL host root; investigate why it needs this

Two things want planning before you roll this across a fleet. The subordinate UID and GID ranges have to exist on every host you enable it on, which makes it a config-management job rather than a one-off on the box you were logged into. And every volume the workloads already depend on needs its ownership looked at before the restart, not after the pager goes off.

Opting a single container out should be rare, and it should leave a paper trail. A --userns=host buried in a compose file or a systemd unit quietly rebuilds the old root-equals-root world for that one workload, usually for a reason nobody wrote down. Make someone say out loud in review why this container needs it.

Rootless mode and userns-remap attack the same assumption from opposite ends. Rootless runs the whole daemon as an ordinary user. Remap keeps the daemon as root and shrinks the containers instead. Running half of each across a fleet gives you neither. Pick one per class of host, write down which one, and make the choice something a script can check.

Treat the two checks above as change-window work. After anyone touches the daemon, run them again, paste the command and its output into the ticket, and refuse to close the change if the reading has moved. Controls like this rarely get switched off deliberately. They get switched off by a rushed rollback at 2am, and they stay off because nobody went back to look.

Be honest about what the mapping does not cover. Every container on the daemon shares that one range, so container root over here is the same host user as container root over there; remap stops a container reaching the host, not a container reaching its neighbor's files. It does nothing about a kernel bug that ignores identity altogether. And it does nothing for anyone who can reach the Docker socket, because that person asks the daemon, which is still root, to do the work on their behalf.

Try this

On a lab host that already has userns-remap configured, start something as container root and then look at the same process from the host side. The UID you get back should sit inside the subordinate range, nowhere near 0.

$ grep -E 'userns|dockremap' /etc/docker/daemon.json /etc/subuid 2>/dev/null | head -5
/etc/subuid:dockremap:100000:65536
$ docker run -d --name remapdemo alpine sleep 120
$ pid=$(docker inspect -f '{{.State.Pid}}' remapdemo)
$ ps -o user,uid,pid,args -p "$pid"
USER       UID   PID COMMAND
100999  100999  ... sleep 120
$ docker rm -f remapdemo

Takeaway

Map container root onto an unprivileged host UID and the classic root-equals-root escapes lose the thing they were standing on. Keep opt-outs rare and reviewed, and when a volume breaks, fix its ownership instead of switching the mapping off.

Related