Why Terragrunt exists

The DRY and state problems it solves.

Thirty-one warehouses, and on the wall of each one a handwritten sticky note tells staff which safe to lock the master ledger in. Someone copies that note by hand every time a warehouse opens. Thirty say safe 4B: fireproof, bolted down, checked weekly. One says 4C. Safe 4C sits in the loading bay with the door off, and nobody notices for a year. That sticky note is Terraform's backend block, the ledger is Terraform state, and the year nobody noticed is the part worth worrying about.

Terragrunt is a thin wrapper around OpenTofu and Terraform, written by a company called Gruntwork. (OpenTofu is the open-source fork of Terraform; the two still read almost identical configuration.) Terragrunt does not replace either one. It runs them. Type a Terragrunt command and it reads its own configuration file, works out the repetitive settings on your behalf, writes a few files to disk, then hands off to the tofu or terraform binary sitting on your PATH (the list of directories your shell searches when you type a program name). You pick which binary with the --tf-path flag or the TG_TF_PATH environment variable. Everything Terraform already did still happens, unchanged. Terragrunt only decides what gets handed to it, and in what order.

Two problems turn up the moment a codebase grows past a handful of directories. First, repetition. Every root module needs its own backend settings, its own provider block, and usually the same handful of input values, copied across every environment and every component. Copies drift, quietly. Second, ordering. Terraform can sequence resources inside one state file perfectly well, but it has no idea how to apply twenty separate state files in dependency order, or how to feed one unit's outputs into the next. Terragrunt answers the first with DRY configuration (Don't Repeat Yourself: write a thing once, point at it from everywhere) and the second with a dependency graph it works out for you.

The Ledger You Cannot Address With a Variable

Terraform state is a JSON file (JavaScript Object Notation, a plain-text format for structured data) that records every real resource Terraform created and maps each one back to the code that asked for it. That is the ledger. It is also a plain-text copy of a great many of your secrets, which people forget with impressive regularity: database passwords, generated private keys, session tokens, any sensitive value a provider handed back. Marking an output as sensitive hides it from your terminal. It does nothing at all to the state file. Anyone who can read that file reads every line of it.

The backend block is the ledger's address. Which bucket in S3 (Simple Storage Service, Amazon's object store), which object key inside that bucket, which region, whether it is encrypted at rest, and how it gets locked so two engineers cannot write over each other in the same minute. In plain Terraform that block lives inside the root module it belongs to. One copy per directory, forever.

terraform {
  backend "s3" {
    bucket  = "acme-tfstate"
    key     = "prod/payments/terraform.tfstate"
    region  = "eu-west-1"
    encrypt = true
  }
}

The obvious fix is to lift the bucket name into a variable and share it. Try that and Terraform stops you at the door.

sed -i 's/"acme-tfstate"/var.state_bucket/' backend.tf
tofu init

╷
│ Error: Variables not allowed
│ 
│   on backend.tf line 3, in terraform:
│    3:     bucket  = var.state_bucket
│ 
│ Variables may not be used here.
╵

Nobody forgot to implement this. It is a chicken-and-egg problem. Terraform has to know where the state lives before it can evaluate anything else, and evaluating variables, locals and data sources needs the state it has not fetched yet. So the backend block is read early and read literally: no variables, no locals, no functions, no data lookups. Partial configuration (leaving values out of the block and passing them at init time with -backend-config) does work, but it shoves the duplication into a wrapper script and a pile of .tfbackend files, and now the settings that matter live somewhere your reviewers never open. That one constraint is the technical reason Terragrunt exists.

What Thirty Copies Look Like When One Drifts

Duplication is a detection problem, not a tidiness complaint. Here are three checks worth running against any Terraform estate you inherit. The middle one uses grep -L, which prints the files that do NOT contain a match, the mirror image of what grep normally hands you.

find live -name backend.tf | wc -l
grep -rL --include=backend.tf 'encrypt' live
grep -rh --include=backend.tf 'bucket' live | sort | uniq -c

31
live/staging/analytics/backend.tf
     30     bucket  = "acme-tfstate"
      1     bucket  = "acme-tfstate-backup-2023"

Thirty-one units. One never asks for encryption. And one writes its ledger into a bucket nobody else uses. Take the missing encrypt line first, because the honest version is less dramatic than the scary version you have probably read elsewhere. encrypt = true tells the backend to send a server-side encryption header when it uploads state. Since January 2023, S3 encrypts every new object by default anyway, so that file is not lying around in the clear on a disk in Dublin. What the missing line actually tells you is that this directory was copied from an older sibling before the line existed, and that nobody read it for a year. It bites in two specific places: if your bucket policy denies uploads that arrive without that header, this unit's apply fails at the worst possible moment, and if the rest of the estate names a customer-managed key with kms_key_id (Key Management Service, where AWS keeps keys whose use you restrict with a policy of their own), this one unit falls back to whatever the bucket default happens to be.

The bucket count is the finding that should get you out of your chair. One unit ships its state somewhere else. Maybe someone tested a migration in 2023 and walked away. Maybe that bucket lives in an account you do not control, in which case a single line is exfiltration wearing the costume of boilerplate, and it reviewed as three ordinary characters in an ordinary file. Keep one thing straight while you clean any of this up: encryption at rest does not stop a reader. S3 decrypts the object on the way out, so anybody holding s3:GetObject (plus permission to use the key, where a customer-managed one is involved) reads the analytics database password in plain text without touching a single running server. Encryption protects the disks and the backups. Permissions protect the secret.

Define It Once, Generate It Everywhere

Terragrunt's move is to stop treating backend.tf as source code and start treating it as build output, the way you treat a compiled binary. You describe the address once, in a root file written in HCL (HashiCorp Configuration Language, the same syntax your .tf files already use), and Terragrunt writes the real file fresh before every run. The file everyone used to copy no longer exists in the repository.

locals {
  region = "eu-west-1"
  bucket = "acme-tfstate"
}
 
remote_state {
  backend = "s3"
  generate = {
    path      = "backend.tf"
    if_exists = "overwrite_terragrunt"
  }
  config = {
    bucket       = local.bucket
    key          = "${path_relative_to_include()}/terraform.tfstate"
    region       = local.region
    encrypt      = true
    use_lockfile = true
  }
}
 
generate "provider" {
  path      = "provider.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<EOF
provider "aws" {
  region              = "${local.region}"
  allowed_account_ids = ["111122223333"]
  default_tags {
    tags = { managed_by = "terragrunt" }
  }
}
EOF
}
 
inputs = {
  region = local.region
}

Read that as three promises rather than three blocks. The key line calls path_relative_to_include(), a Terragrunt function that returns the current unit's path relative to the file it includes, so every unit lands on its own object key and no human ever picks one by hand. use_lockfile = true asks S3 to do the locking itself with conditional writes, which current OpenTofu and Terraform both support, replacing the separate DynamoDB lock table most estates still carry around. allowed_account_ids is an AWS provider argument that aborts the run when the credentials in play belong to an account other than the one named, so a terminal still holding last week's exported keys cannot apply this tree by accident. Every unit gets all three. No unit can quietly opt out, because no unit writes those files.

include "root" {
  path = find_in_parent_folders("root.hcl")
}
 
terraform {
  source = "git::ssh://[email protected]/acme/tf-modules.git//rds?ref=v3.2.1"
}
 
dependency "vpc" {
  config_path = "../vpc"
 
  mock_outputs = {
    vpc_id             = "vpc-00000000000000000"
    private_subnet_ids = ["subnet-0000000000000000a", "subnet-0000000000000000b"]
  }
  mock_outputs_allowed_terraform_commands = ["validate", "plan"]
}
 
inputs = {
  vpc_id         = dependency.vpc.outputs.vpc_id
  subnet_ids     = dependency.vpc.outputs.private_subnet_ids
  instance_class = "db.t4g.medium"
}

That is the whole unit, and not one line of it repeats an address. find_in_parent_folders("root.hcl") walks up the directory tree until it finds the root file, so moving this directory does not break it. In the source line, the double slash marks where the repository ends and the subdirectory inside it begins, and ?ref=v3.2.1 pins an immutable tag. Pin to a tag or a commit hash, never a branch. A branch reference means whoever can push to that branch decides what your next apply builds, in your account, with your credentials. The dependency block points at the VPC unit next door (Virtual Private Cloud, your own private network inside AWS) and pulls its outputs in as inputs here.

Checking That the Generated File Is the File You Meant

Generation is only worth trusting if you can look at the result. Run the unit once, then read what Terragrunt actually wrote.

cd live/prod/payments
terragrunt init > /dev/null
find .terragrunt-cache -name backend.tf -exec cat {} +

# Generated by Terragrunt. Sig: nIlQXj57tbuaRZEa
terraform {
  backend "s3" {
    bucket       = "acme-tfstate"
    encrypt      = true
    key          = "prod/payments/terraform.tfstate"
    region       = "eu-west-1"
    use_lockfile = true
  }
}

That first line is the signature Terragrunt stamps on everything it writes, and it is exactly how if_exists = "overwrite_terragrunt" tells its own files apart from yours: it replaces a file carrying that signature and errors out on one that does not, instead of eating your work. (Plain "overwrite" clobbers either kind without asking, which is the setting people regret.) Because this unit sets a remote source, Terragrunt cloned the module into .terragrunt-cache, a working directory it keeps beside the unit, and wrote backend.tf and provider.tf in there before calling tofu. Units built from local .tf files get the generated files written right next to them instead. Either way, the file open in your editor and the file tofu reads are not the same file, which is the single most disorienting thing about your first week with Terragrunt.

Many Small Ledgers, on Purpose

The layout Terragrunt pushes you toward is many small independent state files, one per component per environment. A vpc unit, a database unit, an app unit, each existing separately in dev, staging and prod. One enormous state file holding everything has three costs that compound as you grow. Every plan refreshes the whole estate, so plans crawl. Every apply can touch anything, so a careless module bump in a test service can roll straight through production networking. And everyone who needs to change anything needs write access to the file that holds everything.

Splitting the state splits the permissions with it. Because each unit writes to its own object key, an IAM policy (Identity and Access Management, the AWS system that decides who may do what) can hand the payments team arn:aws:s3:::acme-tfstate/prod/payments/* and nothing else. That string is an ARN (Amazon Resource Name, AWS's way of writing the full address of a thing). One leaked read-only credential then yields one component's secrets rather than the whole company's. You can do precisely this in plain Terraform, but only if you are willing to hand-maintain a backend block per unit, and you already know how that story ends.

There is a review benefit sitting underneath all of it. In a copy-paste layout, a pull request that changes one directory's bucket name reads as three ordinary lines in an ordinary file, which is exactly how the backup-2023 bucket got there and stayed. When the backend is generated, no unit directory contains a backend block at all, so redirecting state has to touch root.hcl, the one file everybody watches. Put root.hcl behind a required review in your CODEOWNERS file (the file that tells your git host who must approve changes to which paths) and state exfiltration stops being a quiet diff.

Getting the Order Right

You cannot ice a cake before it is baked, and you cannot create a database subnet group before the network exists. Inside a single state file, Terraform sorts that out on its own. Across separate state files it has no idea, and the traditional answer is a runbook telling humans which directory to apply first. Terragrunt reads the dependency blocks instead and builds a DAG (directed acyclic graph: a to-do list where some jobs cannot start until others finish, and nothing is ever allowed to wait on itself). Ask it what it worked out.

cd live && terragrunt find --dag

prod/vpc
prod/payments
prod/api

Same graph, machine readable, in DOT (the plain-text graph language Graphviz reads), which you can pipe straight into dot -Tpng for an architecture picture that cannot go stale.

terragrunt dag graph

digraph {
  "prod/api" ;
  "prod/api" -> "prod/payments";
  "prod/api" -> "prod/vpc";
  "prod/payments" ;
  "prod/payments" -> "prod/vpc";
  "prod/vpc" ;
}

Destruction runs the same graph backwards, and backwards is the direction that ends careers. Ask before you act.

terragrunt find --dag --queue-construct-as=destroy

prod/api
prod/payments
prod/vpc

You Are Still Debugging Tofu

Terragrunt hides none of Terraform. State, plan and apply, providers, modules, lock files, all of it still applies, and when a run fails the error nearly always came out of tofu and was printed through Terragrunt on its way to you. Read the error at the layer that produced it. When a plan makes no sense, cd into the .terragrunt-cache working directory and run tofu there by hand against the generated files, which is the fastest way to learn whether the problem is yours or the wrapper's. Two Terragrunt-specific traps are worth knowing in week one. mock_outputs feeds invented values into a plan when the upstream unit has never been applied, which is what makes a first-ever plan of a whole stack possible, and also what makes that plan a partial work of fiction: fence them in with mock_outputs_allowed_terraform_commands so a real apply can never consume a fake subnet ID. And --dependency-fetch-output-from-state reads an upstream unit's outputs straight out of its state file rather than shelling out to tofu output, which is far quicker on a large stack and quietly needs read access to that other unit's state, so grant it on purpose rather than by accident.

Make the Guardrail Testable

Generated files belong in .gitignore, both so nobody edits them and so a committed one stands out as an anomaly.

# written by terragrunt before every run, never by a human
backend.tf
provider.tf
.terragrunt-cache/

Then put the check in CI (continuous integration, the automation that runs on every pull request) so the old habit cannot creep back in. Alongside it, terragrunt hcl fmt --check and terragrunt hcl validate keep the configuration itself honest, and terragrunt backend bootstrap creates the state bucket with versioning, encryption, TLS-only access and access logging already switched on. Run that bootstrap deliberately: Terragrunt no longer conjures missing backend resources mid-run, so nothing gets provisioned behind your back unless you pass --backend-bootstrap yourself.

grep -rl --include='*.tf' 'backend "s3"' live/ && { echo 'hand-written backend block committed'; exit 1; }

live/staging/analytics/backend.tf
hand-written backend block committed

That is the drifted warehouse from the top of this lesson, caught by a pipeline instead of by a breach report. One line, one second.

Run terragrunt find --dag against your live directory today and read the list against the map of the estate you carry in your head. The units nobody remembers owning are the ones still holding a hand-written backend block.

Try this

Run sed -i 's/"acme-tfstate"/var.state_bucket/' backend.tf on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: run --all Never Stops to Ask. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related