Recipe: Redis

Persistence, memory limits, and auth.

Redis (Remote Dictionary Server) keeps everything in memory, which makes it very fast and very easy to start. Type redis-server with no options and it runs. No password. No limit on how much memory it eats. A snapshot of your data written into whatever folder you happened to start it from, on a schedule you never picked. On your laptop that is fine. Put the same thing on a server with a public address and one safety catch is holding the door. It is called protected mode, it has been on by default since Redis 3.2, and on Redis 7 it turns away every connection that did not arrive from the machine itself for as long as the default user has no password. That includes a connection arriving through a published Docker port. Think of protected mode as a seatbelt: worth having, and no substitute for locking the car. Give the server a password and protected mode steps aside, which is what you want, because a real lock has taken its place. Put protected-mode no in the config instead, as plenty of images and tutorials quietly do, and it steps aside with nothing behind it: an unlocked filing cabinet that anyone can read, empty, or use to write files wherever the server can write. That is inside the container, and it only reaches the host when something bridges the gap: a bind mount of a host path, host networking, or a privileged container. The fix is four short steps. Give Redis a password. Cap its memory. Decide what happens to your data when the container restarts. Keep it off any public port.

The config file

Three of those four fit in a config file five lines long. The memory cap and the persistence choice go in as plain settings. The password goes in by reference: the last line is an include pointing at a small file that Compose hands to the container at startup, so the config file you commit stays free of secrets and the password never has to travel on a command line. That password file still sits on disk, so it needs a .gitignore entry and a folder nobody else can open, and we set both up below. The fourth step is not written down anywhere. Keeping Redis off a public port is a line you leave out of the Compose file, which makes it a strange thing to review and an easy thing to undo by accident.

maxmemory 256mb
maxmemory-policy allkeys-lru
appendonly yes
save ""
include /run/secrets/redis_pw.conf

maxmemory 256mb is the ceiling. Leave it out and Redis keeps asking the kernel for more room until the Linux OOM (out-of-memory) killer, the part of the kernel that starts shooting processes when RAM runs dry, picks a victim. Sometimes the victim is Redis. Sometimes it is the neighbour process you cared about more. maxmemory-policy allkeys-lru says what Redis does once it touches that ceiling. It works like a full club with a one-in-one-out door: the bouncer clears out whoever has gone longest without ordering anything, not whoever walked in first. LRU stands for least recently used, and those are exactly the keys that get thrown out, so new writes keep succeeding instead of failing. Redis does not keep a perfect ranking, either. It samples a handful of keys on each eviction, five by default and tunable with maxmemory-samples, and drops the least recently used one it looked at, which is close enough and far cheaper than ordering every key in the database. appendonly yes turns on the write-to-disk mode we come back to below. save "" switches off the timed snapshots, which are on even when you hand Redis no config at all: the built-in schedule saves after an hour if a single key changed, after five minutes if a hundred did, and after a minute if ten thousand did. Switching them off keeps this instance from writing the same data to disk on two separate schedules. Snapshots are not a legacy format, though: RDB is still the default in the official image, and it is what replication and most backup tooling move around. The saving is smaller than it looks, too, because aof-use-rdb-preamble yes is the default and every rewrite of the journal writes its base in RDB format anyway. Redis itself recommends running both when the data genuinely matters, so treat this line as a cache decision. On a database you would leave the snapshots on.

The Compose stack

Here is a whole stack you can drop into an empty folder and run. The password lives in secrets/redis_pw.conf, which Compose mounts into the container as a secret for redis.conf to include, so it never gets baked into the image and never lands in the file you commit. Now look at what is not there. No ports: block. That absence is deliberate.

services:
  redis:
    image: redis:7-alpine
    command: ["redis-server", "/usr/local/etc/redis/redis.conf"]
    restart: unless-stopped
    mem_limit: 512m
    volumes:
      - ./redis.conf:/usr/local/etc/redis/redis.conf:ro
      - redisdata:/data
    secrets:
      - source: redis_pw
        target: redis_pw.conf
    healthcheck:
      test: ["CMD-SHELL", "REDISCLI_AUTH=\"$$(cut -d' ' -f2 /run/secrets/redis_pw.conf)\" redis-cli ping | grep -q PONG"]
      interval: 10s
      timeout: 3s
      retries: 5
      start_period: 30s
 
volumes:
  redisdata:
 
secrets:
  redis_pw:
    file: ./secrets/redis_pw.conf

The command line starts with the word redis-server, and it has to. The image's entrypoint drops from root to the redis user only when the first argument is exactly redis-server. It does put redis-server in front for you when the first argument starts with a dash, so a command of ["--maxmemory", "256mb"] works too, but a path to a config file matches neither test: hand the entrypoint one on its own and it tries to execute the .conf file, and the container exits before Redis ever starts. Wrap the command in sh -c to read the password inline, the way plenty of examples do, and the first argument is sh: the test fails, the drop is skipped, and the server runs as root, leaving root-owned files behind in the volume. Passing the password in through include means no shell is needed, so the entrypoint does its job and the password never lands in the redis-server arguments where any ps on the host would show it. The redisdata volume mounted at /data is where the append-only file lands, so your data survives the container being thrown away and rebuilt. mem_limit: 512m is the container's own ceiling, set at twice the 256mb Redis ceiling because what the process occupies is always more than the data it holds, which the field notes below get into. The healthcheck logs in with the password and sends a ping every ten seconds. start_period: 30s is there because a large append-only file has to be replayed before Redis answers anything, and a reply of LOADING is not PONG: without that grace window the five failed pings in the first fifty seconds would mark a container unhealthy while it is doing exactly what you asked. The check earns its keep when a second service in the same file lists depends_on: with condition: service_healthy under it, because that service then waits for a real PONG instead of starting the moment the Redis container exists. REDISCLI_AUTH hands the password over through the environment, which keeps it out of the argument list there too. The doubled $$ is not a typo. Compose swallows a single $ while parsing the file, so $$(cut ...) is what reaches the shell as $(cut ...). What you buy is a config file that is safe to commit, an image with no secret baked in, and nothing for ps to leak. What is left to guard is the password file sitting on the host, and that is a folder permission, which is the first thing the next section sets.

Bring it up and prove it works

Write the password file, keep it out of git, start the stack, then talk to the server with redis-cli, the small command-line client that ships inside the image. Change that password before you run this, and keep it to one word with no spaces in it: every command here pulls the password out of the file with cut -d' ' -f2, which hands back the second space-separated field and drops the rest, so a passphrase with spaces in it arrives truncated, the healthcheck never goes green, and nothing tells you why. Note too what the chmod locks. It locks the folder. Redis reads that file as the redis user inside the container, so the file itself has to stay readable, and the folder is what keeps everyone else on the host out.

$ mkdir -p secrets
$ echo 'requirepass S3cret-pw-please-change' > secrets/redis_pw.conf
$ chmod 700 secrets
$ echo 'secrets/' >> .gitignore
$ docker compose up -d
[+] Running 3/3
 ✔ Network redis_default      Created
 ✔ Volume redis_redisdata     Created
 ✔ Container redis-redis-1    Started
$ docker compose exec redis sh -c 'REDISCLI_AUTH="$(cut -d" " -f2 /run/secrets/redis_pw.conf)" redis-cli PING'
PONG
$ docker compose exec redis redis-cli PING
(error) NOAUTH Authentication required.

PONG is Redis saying hello back, so the server is up and your password was accepted. The second call sends no password at all and gets NOAUTH. That is the whole point of the exercise: there is no anonymous way in. Now look at what the first call does not contain. Your password is nowhere in the line you typed. The shell that reads it runs inside the container, REDISCLI_AUTH carries it into redis-cli through the environment, and the host's process list sees a file path and nothing else. That is the healthcheck's trick, borrowed. The shorter redis-cli -a "$PW" makes redis-cli warn you every single time, and the warning is earned: the password is then in an argument list that any user on the host can read while the command runs. --no-auth-warning silences the warning and changes nothing else, so the commands below stay with the environment and park the long half of the line in a shell variable.

$ RCLI='REDISCLI_AUTH="$(cut -d" " -f2 /run/secrets/redis_pw.conf)" redis-cli'
$ docker compose exec redis sh -c "$RCLI SET session:42 alive"
OK
$ docker compose exec redis sh -c "$RCLI GET session:42"
"alive"
$ docker compose restart redis
[+] Restarting 1/1
 ✔ Container redis-redis-1  Started
$ docker compose exec redis sh -c "$RCLI GET session:42"
"alive"

SET wrote a key, GET read it back, and the value is still sitting there after a full restart of the container. That is the append-only file being replayed off the volume as Redis starts. Durability here needs both halves, so be careful not to credit either one with the whole job. Destroy the volume and the key goes with it, because the volume is what outlives the container. Switch to appendonly no while save "" is still in the config and the key goes just as thoroughly, off the same untouched volume, because nothing was ever written for it to hold.

RDB vs AOF, and which to pick

Redis writes to disk in two different ways, and the trade-off is the one your text editor makes with autosave. RDB (Redis Database) is a point-in-time snapshot, like saving the whole document every few minutes. The file stays small and restores quickly, but a crash costs you every change made since the last save. AOF (append-only file) behaves more like a change journal, adding a line for each write as it happens, so a crash costs you a second or two at most. That second is a setting, not a law. appendfsync everysec is the default, and it flushes the journal to disk once a second; always flushes on every single write and is far slower; no leaves the timing to the operating system, which can mean half a minute of writes gone. You pay for the journal with a bigger file that has to be replayed at startup, and on Redis 7, the version this recipe pins, it is not one file but a directory at /data/appendonlydir/ holding a base plus the changes since. A pure cache needs neither, because you can rebuild it from wherever the data really lives. For anything you would be annoyed to lose, run AOF on a volume, which is what this recipe does.

Back to that missing ports: block. Publish Redis on 0.0.0.0 (every network interface the host has) with no password and protected mode switched off, and an internet-wide scanner will find it, often within hours of it going live. Inside a Compose stack you gain nothing by publishing, because the other services already reach this container at redis:6379 over the private network Compose built for them. If something outside the stack genuinely has to connect, bind the published port to 127.0.0.1 so only the host itself can use it, keep the password on, and put a tunnel or a proxy in front of it. That bind is a ports: entry reading 127.0.0.1:6379:6379, and the host address in front of the first colon is the whole difference. A bare -p 6379:6379 is where the breach write-ups begin.

Field notes

requirepass hands every client the same password, which is fine while one application is doing the talking. Once a second thing connects, Redis 6 and later let you define ACL (access control list) users instead, so a metrics agent can be given INFO and nothing else while your application keeps its write access.

Give the container limit real headroom over the Redis ceiling. The usual advice runs from one and a half times maxmemory up to two, and this recipe takes the top of that range: 512m against 256mb. What the process occupies is always more than the data it holds. The memory allocator, the library that hands out and takes back chunks of RAM, ends up sitting on gaps too small to reuse. Client output buffers are counted on top of the data. And rewriting the journal starts a second copy of the Redis process to do the work: the two copies share the same memory to begin with, but every block of it that changes while the rewrite runs has to be duplicated so the copy still sees the old version. A rewrite that runs while writes keep arriving, against a dataset already sitting at 256mb, is exactly the case that eats one and a half times and asks for more. Set the container limit to the same 256m and the kernel kills a server that is behaving itself. Whether allkeys-lru or noeviction is correct comes down to one question: is this a cache, or the only copy? And never call something durable because AOF is switched on. Durable means you restored from a backup recently and watched the data come back.

Keep Redis on a private network, and watch two numbers over time. Both come out of INFO, and INFO wants the password like everything else, so it goes in through the container the way the healthcheck does: docker compose exec redis sh -c 'REDISCLI_AUTH="$(cut -d" " -f2 /run/secrets/redis_pw.conf)" redis-cli INFO stats'. Neither number is easy to find in the wall of output plain INFO gives you at 3am, so ask for them by section. The stats section prints evicted_keys: if that is climbing, the 256mb ceiling is smaller than the working set and allkeys-lru is quietly throwing keys away. Swap stats for memory and you get used_memory_rss and mem_fragmentation_ratio: a ratio well above 1.5 means the allocator is sitting on memory the kernel counts against the container while Redis is not using it, which is the headroom in that 512m limit going away.

When you roll this out for real, put the numbers in the ticket: the image tag you started from, the maxmemory and policy values before and after, which host ran the commands, and the PONG and NOAUTH you saw at the end. Raising a memory ceiling again is easy. Undoing an eviction policy after allkeys-lru has already dropped a few thousand keys is not, so copy the evicted_keys count in beside the change, because it is the only record that those keys ever existed.

Try this

Run these in the same folder as the compose.yaml above, with the stack already running. They check four things the first pass did not: the memory ceiling Redis actually loaded, the policy it will apply when it gets there, which user the server process is running as, and whether anything at all is published to the host. A fifth command asks for the config with no password, because CONFIG GET is the one call you would least like a stranger to get an answer to. Where the password is needed it is read inside the container again.

$ RCLI='REDISCLI_AUTH="$(cut -d" " -f2 /run/secrets/redis_pw.conf)" redis-cli'
$ docker compose exec redis sh -c "$RCLI CONFIG GET maxmemory"
1) "maxmemory"
2) "268435456"
$ docker compose exec redis sh -c "$RCLI CONFIG GET maxmemory-policy"
1) "maxmemory-policy"
2) "allkeys-lru"
$ docker compose exec redis redis-cli CONFIG GET maxmemory
(error) NOAUTH Authentication required.
$ docker compose exec redis ps -o user,args | grep redis-server
redis    redis-server *:6379
$ docker compose ps
NAME            IMAGE            COMMAND                  SERVICE   CREATED         STATUS                   PORTS
redis-redis-1   redis:7-alpine   "docker-entrypoint.s…"   redis     2 minutes ago   Up 2 minutes (healthy)   6379/tcp
# STATUS: PASS when all five line up. 268435456 bytes is the 256mb ceiling, the
# policy is the one you chose, the call with no password is refused, the user
# column on the server process says redis, and the PORTS column shows a bare
# 6379/tcp with no arrow in it, which means the port is reachable inside the
# Compose network but published nowhere on the host.

Takeaway

If you carry one thing away from this recipe, carry the line that is not in the Compose file. There is no ports: block, so nothing outside this host can open a socket to the server at all, and the scanners that find unlocked Redis instances have nothing to knock on. Everything else in here is a second line of defence for the day somebody adds it back.

Related