Stacks: deploying a Compose file

One file, a whole app, across nodes.

Hand one person a shopping list and they walk every aisle themselves. Hand the same list to a team and the work splits itself up without anyone being told which aisle to take. A stack is that second version of Compose. On one machine, docker compose up reads your file and brings up a web service, a database and a network on that single box. A stack hands the same file to every node (machine) in the swarm, which is Docker's built-in clustering mode, where a group of machines behaves like one. You write one file, run one command, and Swarm decides which machine runs what.

The command is docker stack deploy. It reads the file and writes what it finds into the cluster's record of how things ought to look: every service, every overlay network, every secret, every config, and the named volumes your services mount. Edit the file, run the same command again, and Swarm compares what is running against what you just handed it, then changes only the parts that moved. Hold onto that. The same file is your first deploy and every update after it. There is no separate apply-the-changes step to remember.

services:
  web:
    image: registry.internal/web:1.4.2
    ports: ["80:3000"]
    networks: [appnet]
    deploy:
      replicas: 3
      update_config: { parallelism: 1, delay: 10s, failure_action: rollback }
      restart_policy: { condition: on-failure }
      resources:
        limits: { cpus: "0.50", memory: 256M }
  db:
    image: postgres:16
    networks: [appnet]
    volumes: ["pgdata:/var/lib/postgresql/data"]
    deploy:
      placement:
        constraints: ["node.labels.disk == ssd"]
networks:
  appnet: { driver: overlay }
volumes:
  pgdata: {}

Read it from the top. web runs three copies of an image that was built somewhere else, sitting behind port 80. db runs Postgres, a database server, keeps its files on a named volume called pgdata so the data outlives the container, and is pinned to machines labelled disk=ssd (solid-state drive, the fast kind). Both services share appnet, an overlay network, meaning a virtual network stretched across hosts so a task on node-1 can reach one on node-3 as if they shared a cable. A task, by the way, is Swarm's word for one running copy of a service. Now notice what is missing. There is no build: line anywhere. Stacks run images, they never build them, so every image has to already exist in a registry (the server that stores images) that all the nodes can reach.

The deploy block

Everything under deploy: is the half of the file only Swarm reads. replicas is how many copies to keep alive. update_config runs the rolling update: one task at a time here, a 10-second pause between them, and an automatic rollback if a new task fails to start. restart_policy brings a task back when it exits. resources caps the CPU (processor) time and memory each task may take. placement decides which machines are even allowed to run the service. Run this same file with plain docker compose up on your laptop and update_config, placement and the rolling-update policy do nothing at all. Those keys mean something to Swarm and nothing to Compose. On one box they are dead weight. In a cluster they are the difference between 'run this container' and 'keep three healthy copies spread across the fleet, and replace them without dropping a request'.

Deploy it

Point the command at the file and give the stack a name. That name gets stamped on the front of everything Swarm creates, so two stacks living on the same cluster never end up fighting over the same object.

$ docker stack deploy -c compose.yml --with-registry-auth payments
Creating network payments_appnet
Creating service payments_db
Creating service payments_web

Every object came back wearing the payments_ prefix. The network is payments_appnet, the services are payments_web and payments_db. A clean return only tells you the request was accepted, so ask the cluster what it actually did with it.

$ docker stack services payments
ID             NAME           MODE         REPLICAS   IMAGE                         PORTS
73f9a1c7d2e0   payments_web   replicated   3/3        registry.internal/web:1.4.2   *:80->3000/tcp
b8d254a6c8e0   payments_db    replicated   1/1        postgres:16

REPLICAS is the column you live by. 3/3 reads as three copies wanted, three copies running. If web still said 0/3 or 1/3 a minute after the deploy, something is stopping tasks from starting, and the next command tells you where to look. It breaks the stack into individual tasks, one row per copy, showing the machine each one landed on and the state it is in.

$ docker stack ps payments
ID             NAME             IMAGE                         NODE     DESIRED STATE   CURRENT STATE
e1a2b3c4d5e6   payments_web.1   registry.internal/web:1.4.2   node-1   Running         Running 40 seconds ago
a2b3c4d5e6f7   payments_web.2   registry.internal/web:1.4.2   node-2   Running         Running 40 seconds ago
b3c4d5e6f7a8   payments_web.3   registry.internal/web:1.4.2   node-3   Running         Running 39 seconds ago
c4d5e6f7a8d9   payments_db.1    postgres:16                   node-2   Running         Running 40 seconds ago

The three web tasks spread themselves over node-1, node-2 and node-3 on their own. You never asked for that. The db task landed on node-2, one of the machines carrying the ssd label its placement constraint demands. That is the constraint doing its job. Nothing got scheduled onto a node missing the label.

Prove it serves traffic, then ship an update

Tasks showing Running is a good sign. It is not proof that the app answers. Every port a stack publishes is reachable through the routing mesh, Swarm's shared front door: knock on port 80 at any node in the cluster and your request gets handed to a healthy web task, even one sitting on a different machine. So point curl, the command-line tool that fetches a URL, at a node and look for the app's own reply rather than an open port.

$ curl -s http://node-1/health
{"status":"ok","version":"1.4.2"}

That one-line reply came back from one of the three web tasks. It proves the image, the port mapping and the overlay network are wired end to end. Containers starting proves none of that on its own. Now ship a change. A new build lands as 1.4.3, so you edit that single line in the file and run the exact same command you ran the first time.

$ docker stack deploy -c compose.yml --with-registry-auth payments
Updating service payments_web (id: 73f9a1c7d2e0)
Updating service payments_db (id: b8d254a6c8e0)

Both services get an update call, but only web's spec actually differs, so only web's tasks cycle. db is left where it is. Watch the rollout swap them one at a time.

$ docker service ps --filter desired-state=running payments_web
ID             NAME             IMAGE                         NODE     DESIRED STATE   CURRENT STATE
a7b8c9d0e1f2   payments_web.1   registry.internal/web:1.4.3   node-1   Running         Running 8 seconds ago
a2b3c4d5e6f7   payments_web.2   registry.internal/web:1.4.2   node-2   Running         Running 6 minutes ago
b3c4d5e6f7a8   payments_web.3   registry.internal/web:1.4.2   node-3   Running         Running 6 minutes ago

One task already reads 1.4.3 while the other two still serve 1.4.2 and wait their turn. parallelism: 1 is why they go one after another instead of all at once, which means a healthy task is always taking traffic while the update runs. A few seconds later all three read 1.4.3.

One file, many machines

docker stack deploy turns a Compose file into a set of Swarm services, networks and secrets living under one stack name. That is how a multi-service app lands on a dozen machines from one command. Not every Compose feature survives the trip, so check your deploy: keys, your placement rules and your secrets before you trust the file. Local docker compose up and stack deploy are cousins, not twins.

Keep stack files in Git so a deploy is a reviewed change instead of somebody's terminal history. Pin images by digest rather than a tag that can move under you, and parameterise the registry so the same file works in staging and in production. Have CI (continuous integration, the pipeline that builds and ships your code automatically) deploy the exact file your operators read. The cost is the odd Compose feature Swarm quietly ignores. The payoff is that anyone can open one file and see the whole application. For anything with more than one service, the stack file wins.

Tearing down is where people lose data. docker stack rm takes the services and the networks with it. Named volumes like pgdata are a different story and can outlive the stack, depending on how they were declared. Work out which of your volumes stay behind before you run the removal, not after, because an empty pgdata looks exactly like the one holding last night's orders.

When you do this on a real cluster, write four things in the ticket: the stack name, the image tag that was running before, the tag you deployed, and the machine you typed the command on. That is your rollback plan. If 1.4.3 misbehaves, you put 1.4.2 back in the file and run the same deploy, and update_config walks it backwards one task at a time exactly the way it walked forwards. The person who wrote that down recovers in a minute. The person trusting their scrollback is still working out which of the three nodes had the good version.

Try this

Run these on a lab engine (Docker 24 or newer is fine). Read the sample output first so you know what a healthy result looks like before you lean on the command anywhere that matters.

$ cat > stack.yml <<'EOF'
version: "3.8"
services:
  web:
    image: nginx:1.27-alpine
    ports: ["8080:80"]
    deploy:
      replicas: 2
EOF
$ docker stack deploy -c stack.yml demo
Creating network demo_default
Creating service demo_web
$ docker stack services demo
ID        NAME       MODE         REPLICAS   IMAGE
…         demo_web   replicated   2/2        nginx:1.27-alpine
# STATUS: SUCCESS — stack services created

Takeaway

Treat the stack file as the thing that is true and the cluster as a copy of it. Every change goes through the file, and docker stack deploy stays the only verb: the first run creates, every run after it compares and patches. The two things that catch people out are private images deployed without --with-registry-auth and named volumes like pgdata outliving the stack that created them.

Related