CoursesAtlantisRunning the server & webhooks

Running the server & webhooks

Wiring Atlantis to your VCS.

Advanced12 min · lesson 2 of 12

Every office building lobby has a concierge desk. The concierge sits at one fixed street address all day, carries a badge that opens every room they look after, and answers a doorbell wired in from the tenants upstairs. Only callers who use the agreed knock get answered, and a guest list decides which tenants the concierge will work for at all. Atlantis is that concierge, and those four things have names. The badge is a token for your version-control system (VCS, the GitHub or GitLab that stores your code). The doorbell is a webhook. The knock is the webhook secret. The guest list is the repo allowlist. Get all four right and the desk runs itself. Get one wrong and the doorbell rings into an empty room.

That is the whole deployment model. Atlantis ships as a single Go binary, or as a container image, and it runs as a long-lived HTTP server (HyperText Transfer Protocol, the request-and-answer language the web is built on). Continuous integration jobs spin up when something triggers them and vanish when they finish. Atlantis does not work that way. It has to be already running and already reachable *before* the first pull request arrives, because every single thing it does begins life as an inbound HTTP request. A webhook is exactly that: an HTTP callback your version-control system fires at a URL you registered with it, whenever something happens in a repo. A push. A pull request opened. A comment posted.

Three wires, and the direction of each one matters

Setup means running three wires. First, a VCS token. Atlantis authenticates *outbound* to GitHub or GitLab so it can clone repos, read pull-request metadata, and post plan output as comments. On GitHub that means a bot user's personal access token (PAT) carrying repo scope, or a GitHub App when you want finer-grained permissions. The *Credentials & secrets* lesson compares the two. Second, the webhook. Your version-control system pushes events *inbound* to Atlantis, at the /events endpoint. Third, cloud credentials, so the Terraform that Atlantis shells out to can actually talk to AWS, Google Cloud or Azure. That one is also covered later. Break the token and Atlantis cannot answer. Break the webhook and it never hears the question. Seen from inside the pull request, both failures look identical: nothing happens.

Diagnosing a silent webhook
Plan comment never appears
Open the webhook's Recent Deliveries page and read the status
No delivery listed
Wrong event types
You didn't subscribe pull_request / push / issue_comment / review
Timeout or refused
Network or URL
DNS, the Ingress, or --atlantis-url doesn't match reality
HTTP 400
Secret mismatch
HMAC signature check fails; the Atlantis log confirms it
HTTP 403 (repo)
Allowlist mismatch
Repo not in --repo-allowlist; usually also commented on the PR
The delivery log and the server log are the only ground truth you have. Every silent failure means one of two things: Atlantis never heard the event, or it heard it and refused to trust it.

Start the server

A workable server needs five settings. Every command-line flag has a matching environment variable with an ATLANTIS_ prefix (--gh-token becomes ATLANTIS_GH_TOKEN), and for anything secret you want the environment variable, so the value never lands in ps output or your shell history. Two of the five deserve a word before you copy anything. --atlantis-url is the public address Atlantis believes it lives at. Atlantis prints that address in its web interface and in the links it posts back to pull requests, so it has to be somewhere your version-control system can genuinely reach. Testing from a laptop means opening a tunnel with something like ngrok, because GitHub has to reach *you*, not the other way round. --data-dir is where the working state sits, defaulting to ~/.atlantis: repo clones, the plan files held between plan and apply, and a small BoltDB database (an embedded key-value store that lives in a single file on disk) holding the locks. That one directory is the reason Atlantis is stateful and cannot be treated as a throwaway container.

terminal
# Every flag maps to an env var: --gh-token -> ATLANTIS_GH_TOKEN.
# Keep secrets in env vars so they never appear in `ps` or shell history.
$ export ATLANTIS_GH_TOKEN="ghp_..." # bot PAT, repo scope
$ export ATLANTIS_GH_WEBHOOK_SECRET="$(openssl rand -hex 32)"
$ atlantis server \
--gh-user=atlantis-bot \
--repo-allowlist="github.com/acme/*" \
--atlantis-url="https://atlantis.acme.com" \
--data-dir="/var/lib/atlantis"
{"level":"info","ts":"2026-07-13T10:15:04.212Z","caller":"server/server.go:447","msg":"Atlantis started - listening on port 4141","json":{}}

Port 4141 is the default, and --port changes it. The server exposes /healthz for health probes, and serves a small web page at the root path listing every active lock. That page has no authentication at all by default. Switch on --web-basic-auth=true together with --web-username and --web-password, and do set real values, because they default to atlantis and atlantis. Or keep the whole interface off the public internet. The *Hardening* lesson takes this further.

Wire up the webhook, then prove it works

On the version-control side you register four things: the payload URL https://<atlantis-url>/events, a content type of application/json, your secret, and exactly four GitHub event types. Each event type buys you one feature. Pull requests tell Atlantis that a pull request opened or closed. Pushes let it re-plan when new commits land. Issue comments matter because atlantis apply arrives as an ordinary comment on the pull request. Pull request reviews feed the approval-based gating. Underneath all of it, GitHub takes each payload and computes an HMAC (hash-based message authentication code, a fingerprint that both sides can calculate from a shared secret but an outsider cannot forge), then ships that fingerprint in the X-Hub-Signature-256 header. Atlantis recomputes the same fingerprint and answers 400 when the two disagree. GitLab is weaker here. It sends the secret itself, verbatim, in an X-Gitlab-Token header, which is a static string comparison rather than a signature computed fresh for every payload.

terminal
# Register the webhook on the repo via the GitHub CLI (works org-wide too).
$ gh api repos/acme/infra/hooks --method POST \
-f name=web -F active=true \
-f "events[]=pull_request" -f "events[]=push" \
-f "events[]=issue_comment" -f "events[]=pull_request_review" \
-f "config[url]=https://atlantis.acme.com/events" \
-f "config[content_type]=json" \
-f "config[secret]=$ATLANTIS_GH_WEBHOOK_SECRET"
{
"id": 4931207,
"active": true,
"events": ["pull_request", "push", "issue_comment", "pull_request_review"]
}
# Fire a test ping at /events, then check the hook's Recent Deliveries -> HTTP 200
$ gh api repos/acme/infra/hooks/4931207/pings --method POST
No webhook secret plus a wildcard allowlist is an open door to your cloud
Anyone who can reach /events can POST a forged issue_comment event carrying the words atlantis apply. Without --gh-webhook-secret, Atlantis has no way to tell that forgery apart from the real thing. The signature check is the *only* proof an event came from your version-control system. Sitting behind a corporate network proves nothing once the URL leaks, and the URL leaks easily, because Atlantis puts it in the links it posts in its own pull-request comments. Pair the secret with a tight --repo-allowlist. github.com/acme/* keeps runs inside your org, while a bare * means any repo, on any host, that can aim a webhook at you gets to drive Terraform against your accounts. Both of these are security controls rather than setup chores, and the secret deserves rotating like any other credential.

In production: the Helm chart on Kubernetes

In production almost nobody runs the bare binary. The official Helm chart is the well-trodden path, and its defaults read like advice. It deploys a StatefulSet (a Kubernetes workload type whose pods keep a stable name and stay attached to the same disk) with a PersistentVolumeClaim (a request for storage that outlives the pod) for the data directory. The reason is blunt. If saved plans and the lock database evaporate every time the pod restarts, every pull request in flight loses its plan halfway through review. The chart also defaults to one replica, because BoltDB accepts a single writer, so you cannot scale out by adding pods and hoping. A Redis locking backend exists (--locking-db-type redis) for high-availability setups, but one pod with persistent storage is the road most teams take.

values.yaml
orgAllowlist: github.com/acme/*
atlantisUrl: https://atlantis.acme.com
github:
user: atlantis-bot
# Pull the token + webhook secret from a pre-created Kubernetes Secret
# (keys: github_token, github_secret) instead of committing them here.
vcsSecretName: atlantis-vcs
volumeClaim:
enabled: true
dataStorage: 8Gi # clones, saved plans, and the lock DB live here
terminal
$ kubectl create ns atlantis
$ kubectl -n atlantis create secret generic atlantis-vcs \
--from-literal=github_token="$ATLANTIS_GH_TOKEN" \
--from-literal=github_secret="$ATLANTIS_GH_WEBHOOK_SECRET"
$ helm repo add runatlantis https://runatlantis.github.io/helm-charts
$ helm install atlantis runatlantis/atlantis -n atlantis -f values.yaml
$ kubectl -n atlantis get pods
NAME READY STATUS RESTARTS AGE
atlantis-0 1/1 Running 0 45s # StatefulSet: note the -0

Put an Ingress in front of the pod and terminate TLS (Transport Layer Security, the encryption behind the s in https) there. Strictly speaking, /events is the only path your version-control system ever needs to reach. Exposing that one path and keeping the web interface internal is a cheap hardening win, and we will make it formal later.

Smoke test, then read the delivery log

Test in the same order the events travel. Is the server alive? Does the doorbell ring? Does a real pull request produce a plan?

terminal
$ curl -s https://atlantis.acme.com/healthz
{
"status": "ok"
}
# Open a trivial PR that touches a .tf file. Within seconds Atlantis comments:
#
# Ran Plan for dir: `.` workflow: `default`
#
# Plan: 1 to add, 0 to change, 0 to destroy.
# * To apply this plan, comment: `atlantis apply -d .`

When that comment never shows up, the webhook's Recent Deliveries page in GitHub is your decision tree. No delivery listed at all? You subscribed the wrong event types. Timeout, or connection refused? DNS (Domain Name System, the internet's address book), the Ingress, or --atlantis-url disagrees with reality. A 400 coming back? The secret does not match, and the Atlantis log will show the payload signature check failing. A 403 whose body mentions a repo that is not allowlisted? Allowlist mismatch. Entries are host-qualified paths with no scheme in front (github.com/acme/infra, wildcards allowed), and a near miss gets rejected. That last one is the exception to the silence, because Atlantis usually says so in the pull request itself, commenting that the repo is not allowlisted, unless you pass --silence-allowlist-errors. Every other failure leaves the pull request quiet, since Atlantis either never heard the event or heard it and refused to trust it. The delivery log and the server log are the only ground truth you get, so ship those server logs somewhere searchable on day one.

The wiring is done. Events arrive signed, runs stay scoped to your repos, and state survives a restart. Next we follow a single pull request the whole way down that pipe, from atlantis plan through review to atlantis apply, and look at what Atlantis is doing in the gaps between the comments.

A classic outage reads like this. Atlantis is up, the dashboard is green, and GitHub has been quietly recording failed deliveries for two days. Plans stopped appearing. Engineers drifted back to applying from their laptops. Nobody noticed until two of those laptop applies collided on a Saturday. Watch delivery status, the TLS certificate on the ingress and its expiry date the same way you watch any public endpoint standing between people and production.

Give Atlantis a dedicated bot account, or a GitHub App, with the smallest set of permissions that still works: read on contents, write on pull requests for comments and statuses, and nothing whatsoever that can alter org settings. Rotate that credential on a date you picked in advance, not on the morning a leak makes the news.

Write the four wires on a sticky note next to the on-call cheatsheet: public URL, webhook secret, VCS token, allowlist. When plans go silent, walk those four before you reinstall the chart. Most tickets that say "Atlantis is broken" turn out to be a rotated secret that never reached the pod, or an allowlist still naming the old org.

Try this

Stand up Atlantis with a webhook secret and a repo allowlist, either in a non-production cluster or in a local Docker Compose stack. Fire a ping from your version-control system's web interface, then confirm the server logs an event it actually trusted.

terminal
kubectl -n atlantis logs deploy/atlantis --tail=50 | grep -i webhook
# or local:
# docker logs atlantis 2>&1 | grep -i "request|secret|allowlist"
output
POST /events 200
# Payload signature verified
# Ignoring push event that did not match allowlist regex
# — wrong secret => 4xx; wrong allowlist => quiet ignore

Takeaway

Atlantis is a long-lived HTTP server with four wires running into it: webhook URL, webhook secret, VCS token, repo allowlist. One loose wire and the desk is empty.

Next: keep the webhook secret and the VCS token out of the image, pin the allowlist to exactly your org, and alert on webhook delivery failures from the Git host's side, which is where those failures actually show up.

Quick check
01A pull request touches a .tf file and Atlantis says nothing. The hook's Recent Deliveries page lists the delivery, and it came back HTTP 400. What does that status rule in?
Incorrect — An allowlist miss answers 403, and Atlantis normally leaves a comment on the pull request naming the repo it would not act on.
Incorrect — Events you never subscribed to leave no row on the deliveries page at all, so there would be no status code sitting there to read.
Correct — 400 is the signature check failing, which means the secret registered on the hook and the secret the pod holds have drifted apart.
Incorrect — A bad address or a broken route shows up as a timeout or a refused connection, not as a status that Atlantis itself chose to send.
02The gh api repos/acme/infra/hooks call subscribes four event types. Take issue_comment out of that list and what stops working?
Correct — Every instruction you give Atlantis is typed as a comment, so dropping that subscription leaves the server deaf to all of them.
Incorrect — Opening and closing ride on pull_request, which is still subscribed, so the first autoplan keeps firing as before.
Incorrect — Re-planning after new commits is driven by push, a separate subscription that survives losing comment events.
Incorrect — A review verdict travels on pull_request_review, so gating is unaffected by whether comment events reach the server.
03No plan comment appears, and Recent Deliveries shows the delivery timing out with no HTTP status ever coming back. Which check do you run first?
Incorrect — A mismatched secret still gets an answer out of the server, and that answer is 400, so a silent timeout is a different fault.
Incorrect — A repo outside the allowlist gets 403 back and usually a comment on the pull request, which is a long way from silence.
Incorrect — A subscription you never made leaves the deliveries page empty for that action, rather than filling it with rows that time out.
Correct — Nothing reached the server, so the fault sits on the path in: name resolution, the Ingress, or a URL nobody outside can dial.

Related