State: the heart of Terraform

How Terraform tracks reality.

A coat check works because of a paper ticket. You hand over your coat, the attendant hangs it on hook 47, and you walk out with ticket 47. Every coat on that rack is a black wool coat. The ticket is the only thing tying you to yours. Burn the ticket book and the rack turns into a pile of anonymous coats that nobody can claim.

Terraform's state file is the ticket book. When Terraform builds something, it writes a line into terraform.tfstate, a JSON (JavaScript Object Notation, plain text arranged as labelled fields) document that maps each address in your code, like aws_instance.web, to the identifier the cloud handed back, like i-0abc123def4567890. That one mapping is why the next run knows those few lines of HCL (HashiCorp Configuration Language, the .tf language you write Terraform in) mean that exact EC2 (Elastic Compute Cloud, Amazon's virtual machine service) instance and not a brand new one.

Here is the part people get wrong. Terraform never asks your cloud account “what did I build last time?” There is no such call. It works from three separate pictures of the world instead. Your code is the desired state. State is what Terraform believes it built. Reality is whatever the provider (the plugin that talks to Amazon Web Services, Azure, or whoever you use) reports when it looks up each recorded ID through the cloud's API (application programming interface, the set of calls that create and read things). A plan is the arithmetic between those three. Nearly every baffling plan you will ever read comes from one of them differing from what you assumed.

What Is Actually Inside the File

{
  "version": 4,
  "terraform_version": "1.13.3",
  "serial": 12,
  "lineage": "8f4c1e2a-9b57-4d3f-a1c6-7e05b2d94831",
  "outputs": {},
  "resources": [
    {
      "mode": "managed",
      "type": "aws_instance",
      "name": "web",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "schema_version": 1,
          "attributes": {
            "id": "i-0abc123def4567890",
            "instance_type": "t3.micro",
            "private_ip": "10.0.1.42",
            "vpc_security_group_ids": ["sg-0a1b2c3d4e5f60718"]
          },
          "sensitive_attributes": [],
          "dependencies": ["aws_security_group.web_sg"]
        }
      ]
    }
  ],
  "check_results": null
}

Four of those fields earn their keep. serial is the page number: it goes up by one on every write, so whatever stores your state can tell whether the copy you are about to save is newer or older than the copy it already holds. lineage is the name stamped on the front of the ledger, a UUID (universally unique identifier, a long random string with no realistic chance of colliding with anyone else's) minted once when the state was born and never changed after, and it is what stops you from pushing your staging records over production's. dependencies remembers what each resource was built on top of, which is how terraform destroy still tears things down in the right order months after you deleted the code that described them. mode says whether the entry is something Terraform manages or a data source (a read-only lookup, like asking AWS for the ID of a network somebody else owns), because those get cached in state too.

Reading State Without Breaking It

$ terraform state list

aws_db_instance.app
aws_instance.web
aws_security_group.web_sg
random_password.db

$ terraform state show aws_instance.web | head -20

# aws_instance.web:
resource "aws_instance" "web" {
    ami                                  = "ami-0e2c8caa4b6378d8c"
    arn                                  = "arn:aws:ec2:eu-west-1:123456789012:instance/i-0abc123def4567890"
    associate_public_ip_address          = false
    availability_zone                    = "eu-west-1a"
    cpu_core_count                       = 1
    cpu_threads_per_core                 = 2
    disable_api_stop                     = false
    disable_api_termination              = false
    ebs_optimized                        = false
    get_password_data                    = false
    hibernation                          = false
    id                                   = "i-0abc123def4567890"
    instance_initiated_shutdown_behavior = "stop"
    instance_state                       = "running"
    instance_type                        = "t3.micro"
    monitoring                           = false
    private_dns                          = "ip-10-0-1-42.eu-west-1.compute.internal"
    private_ip                           = "10.0.1.42"

Three commands cover most days. terraform state list prints every address Terraform tracks. terraform state show <address> prints the recorded attributes for one of them. terraform show prints the whole state in that same readable form, and terraform show -json prints it as machine-readable JSON for scripts. When a plan wants to change something and you cannot work out why, put your code next to terraform state show for that address and read the two side by side. The answer is nearly always sitting in the difference. One more is worth learning: terraform state pull fetches the raw state from wherever it lives, local file or remote bucket, and prints it to standard output, which makes it the right way to poke at remote state with jq (a command-line tool for slicing JSON) without downloading anything.

State Holds Your Secrets in the Clear

The ticket book has a second column nobody warns you about. When a provider creates a resource, it writes down everything the cloud returned plus everything you passed in. Generate a database password inside a module and that password lands in state as readable text. Same story for a generated TLS (Transport Layer Security, the encryption behind HTTPS) private key, the contents of a Kubernetes Secret, a service account key, an access token some resource minted on your behalf. Terraform encrypts none of it. (OpenTofu, the open-source fork, added client-side state encryption in 1.7. Terraform itself still leans entirely on whatever the storage underneath provides.) The file is exactly as sensitive as the most sensitive thing inside it, which on a real stack usually means production database credentials.

$ terraform state pull | jq '.resources[] | select(.type == "random_password") | .instances[0].attributes'

{
  "bcrypt_hash": "$2a$10$3vN1qk8Yd0ZC7pO9sIu1WeQ2hR5tX4bV6mA8nJ0lK3fG7yD1cS5eu",
  "id": "none",
  "keepers": null,
  "length": 24,
  "lower": true,
  "min_lower": 0,
  "min_numeric": 0,
  "min_special": 0,
  "min_upper": 0,
  "number": true,
  "numeric": true,
  "override_special": null,
  "result": "Qx7tR2vP9wLmZ4dHs1Ub6Ej0",
  "special": false,
  "upper": true
}

Marking a variable or an output sensitive = true changes what your terminal prints. It does not encrypt, mask, or remove one byte of the file.

$ terraform output db_password

<sensitive>

$ terraform output -raw db_password

Qx7tR2vP9wLmZ4dHs1Ub6Ej0

State held that password in the clear the whole time, either way. Saved plans behave the same. terraform plan -out=tfplan writes a binary file carrying the values it is about to use, so a CI (continuous integration, the system that builds and deploys on every push) pipeline that publishes tfplan as a build artifact is publishing your credentials to everybody who can read builds.

$ ls -l terraform.tfstate*

-rw-r--r-- 1 deploy deploy 18452 Jul 21 09:14 terraform.tfstate
-rw-r--r-- 1 deploy deploy 17903 Jul 21 09:02 terraform.tfstate.backup

Check those permissions on any host where real applies run. -rw-r--r-- means every local account can read the file, and on a shared build box that is one stray shell away from a full credential dump. terraform.tfstate.backup is the previous version, written automatically before each change, holding the same secrets, and it is the file people forget when they write an ignore rule. One more that trips everybody: with a remote backend configured, .terraform/terraform.tfstate is a tiny local file recording which backend you point at, not your resources. Run chmod 600 on both state files now. The lasting fix is a remote backend with encryption at rest and real access control, which is a lesson of its own.

# provider plugins and the backend pointer
**/.terraform/*
 
# state, its automatic backup, and per-workspace state
*.tfstate
*.tfstate.*
terraform.tfstate.d/
 
# saved plans carry the same secrets as state
*.tfplan
tfplan
 
# variable files often hold credentials
*.tfvars
*.tfvars.json
 
crash.log
crash.*.log
 
# .terraform.lock.hcl is deliberately absent from this list: commit it.

When Reality Stops Matching the Ledger

Come back to the coat rack in the morning and one coat is missing, its ticket still sitting in the book. That gap between the ledger and the room is drift: anything the cloud holds that state does not, opened up by something other than Terraform. A person clicking through the console at 3am during an incident. A script running with an admin key. An attacker who got hold of credentials and widened a firewall rule. Terraform gives you a way to stare at that gap without offering to close it. A refresh-only plan re-reads every recorded resource through the provider and reports what moved.

$ terraform plan -refresh-only

random_password.db: Refreshing state... [id=none]
aws_security_group.web_sg: Refreshing state... [id=sg-0a1b2c3d4e5f60718]
aws_instance.web: Refreshing state... [id=i-0abc123def4567890]
aws_db_instance.app: Refreshing state... [id=app-prod-db]
 
Note: Objects have changed outside of Terraform
 
Terraform detected the following changes made outside of Terraform since the
last "terraform apply" which may have affected this plan:
 
  # aws_security_group.web_sg has been changed
  ~ resource "aws_security_group" "web_sg" {
        id      = "sg-0a1b2c3d4e5f60718"
      ~ ingress = [
          + {
              + cidr_blocks      = [
                  + "0.0.0.0/0",
                ]
              + description      = ""
              + from_port        = 22
              + ipv6_cidr_blocks = []
              + prefix_list_ids  = []
              + protocol         = "tcp"
              + security_groups  = []
              + self             = false
              + to_port          = 22
            },
            # (1 unchanged element hidden)
        ]
        name    = "web-sg"
        # (7 unchanged attributes hidden)
    }
 
 
This is a refresh-only plan, so Terraform will not take any actions to undo
these. If you were expecting these changes then you can apply this plan to
record the updated values in the Terraform state without changing any remote
objects.

Somebody opened SSH (Secure Shell, the protocol you log into servers with) to 0.0.0.0/0, which is shorthand for the entire internet, on a production security group. Terraform spotted it because the provider read that group and handed back an ingress rule state had never recorded. Now watch what the two modes do with the same fact. A normal terraform plan offers to delete the rule, since your code is the desired state and the rule is not in it. terraform apply -refresh-only offers the opposite: leave the cloud alone and update state to admit the rule exists. Pick deliberately. Reverting an intruder's change also wipes the evidence of it, so capture the rule, the CloudTrail event (CloudTrail is AWS's log of every API call made in the account), and the timestamp before you let any plan tidy it away.

You do not want to catch this by hand. Put the check on a schedule and let exit codes do the talking. terraform plan -detailed-exitcode returns 0 for no changes, 1 for an error, and 2 when the plan is not empty. On a repository whose main branch is always applied, a 2 means something moved underneath you.

#!/usr/bin/env bash
# Nightly drift check. Terraform exit code 2 means "the plan is not empty".
set -uo pipefail          # NOT -e: an exit code of 2 is a result, not a failure
 
dir=/srv/infra/prod
log=/var/log/tf-drift/plan.txt    # created by LogsDirectory= in the unit
 
cd "$dir" || { logger -t tf-drift -p daemon.err "cannot cd to $dir"; exit 1; }
 
terraform plan -detailed-exitcode -input=false -lock=false -no-color > "$log"
rc=$?
 
case "$rc" in
  0) logger -t tf-drift -p daemon.info    "prod still matches state" ;;
  2) logger -t tf-drift -p daemon.warning "DRIFT in prod, see $log" ;;
  *) logger -t tf-drift -p daemon.err     "plan failed (rc=$rc)"; exit 1 ;;
esac

[Unit]
Description=Terraform drift check for prod
Wants=network-online.target
After=network-online.target
 
[Service]
Type=oneshot
User=terraform
WorkingDirectory=/srv/infra/prod
Environment=TF_IN_AUTOMATION=1
ExecStart=/usr/local/bin/tf-drift.sh
 
# the plan text can echo real attribute values, so keep it to one user
LogsDirectory=tf-drift
LogsDirectoryMode=0700
 
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ReadWritePaths=/srv/infra/prod

[Unit]
Description=Run the Terraform drift check nightly
 
[Timer]
OnCalendar=*-*-* 03:15:00
RandomizedDelaySec=15m
Persistent=true
 
[Install]
WantedBy=timers.target

$ sudo systemctl daemon-reload
$ sudo systemctl enable --now tf-drift.timer
$ systemctl list-timers tf-drift.timer

Created symlink /etc/systemd/system/timers.target.wants/tf-drift.timer → /etc/systemd/system/tf-drift.timer.
NEXT                        LEFT     LAST PASSED UNIT           ACTIVATES
Wed 2026-07-22 03:21:07 UTC 17h left n/a  n/a    tf-drift.timer tf-drift.service
 
1 timers listed.
Pass --all to see loaded but inactive timers, too.

Two honest limits before you trust that timer. First, the host now holds cloud credentials and read access to state, so it is production infrastructure and deserves the same hardening as anything else in prod, not the treatment a spare utility box gets. The unit above assumes the machine gets its credentials from an instance role rather than files in a home directory, which is what lets ProtectSystem=strict stay switched on. Second, and this one matters more: a drift check only sees resources that are already in state. If an attacker creates a fresh IAM (Identity and Access Management, the AWS service holding users, roles, and permissions) user, a new instance, an entire extra network, Terraform holds no ticket for any of it and will never mention it. Plans tell you about things you manage. Finding things you do not manage is a job for CloudTrail, Cloud Audit Logs, or a cloud inventory tool. Run both, and read journalctl -t tf-drift when the alert fires.

Changing State on Purpose

Sooner or later you rename a resource in your code. Terraform reads that as two separate facts: aws_instance.web is gone, aws_instance.web_server is new. Same machine, new address, and a plan offering to replace your production instance over a cosmetic edit. The fix is to move the ticket, not the coat.

$ terraform state mv aws_instance.web aws_instance.web_server

Move "aws_instance.web" to "aws_instance.web_server"
Successfully moved 1 object(s).

That works. There is a better version of it. A moved block does the same rename but lives in your code, gets read in a pull request, and applies itself for everyone on the next run instead of relying on each engineer remembering to type a command. Adopting resources that already exist follows the same shape: an import block is the reviewable form of terraform import, and terraform plan -generate-config-out=legacy.tf will even write you a starting configuration for whatever it finds.

moved {
  from = aws_instance.web
  to   = aws_instance.web_server
}
 
import {
  to = aws_security_group.legacy
  id = "sg-0f9e8d7c6b5a41320"
}

Two more verbs finish the set. terraform state rm makes Terraform forget a resource while the resource carries on running, which is how you hand something over to a different configuration: forget it here, import it there. And terraform apply -replace=aws_instance.web forces one resource to be destroyed and rebuilt on the next apply. It took over from terraform taint, which has been deprecated since 0.15.2. Both edit the ledger, so both deserve the same review as a code change.

The First Check to Run on a Stack You Inherit

$ git log --all --full-history --oneline -- '*.tfstate' '*.tfstate.*'

9f3c2ab bootstrap prod networking

One line of output is one line too many. That commit put every password, key, and token that was in state into every clone of the repository, and deleting the file in a later commit did not take the blob (Git's stored copy of the file contents) out of history. Rotate what leaked, then fix the ignore rules. If the command prints nothing, run terraform plan -refresh-only next and read what the previous owner left behind before you apply anything of your own.

Try this

Run terraform state list on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: state and plan files never belong in Git. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related