Remote state & backends

State for teams, locked and shared.

A local state file is a paper ledger in your own desk drawer. State is OpenTofu's record of what it built and what it believes each thing looks like right now. A private ledger works fine while you are the only person touching the building. The moment a second engineer starts keeping their own copy, you have two ledgers that each claim to be the truth, and neither one knows the other exists. Two people run tofu apply from private snapshots, and whoever writes last wins. Resources get duplicated, orphaned, or quietly destroyed. A remote backend moves the ledger into a shared safe: one authoritative copy, versioned, with a single pen chained to it so exactly one person can write at a time. The plan and apply loop you already know does not change. A backend changes where state lives, who can read it, and who is allowed to write.

What You Are Actually Moving

Before you move state anywhere, look hard at what you are moving. State is a JSON (JavaScript Object Notation, a plain-text format for structured data) document that records every resource OpenTofu manages, the real identifier the cloud handed back, and every attribute the provider returned. That last part is the one people skip past. Attributes include the values you marked sensitive in your code: a generated database password, a private key, an access key OpenTofu created on your behalf. They all land in state as readable text. You can see it yourself with jq, a small command-line tool for pulling fields out of JSON.

# how big is the ledger, and what is sitting inside it?
jq -r '.version, .lineage, (.resources | length)' terraform.tfstate
 
# any password the random provider generated for you
jq -r '.resources[] | select(.type == "random_password") | .instances[].attributes.result' terraform.tfstate

4
6cd1e0b6-2b9a-4a01-9b8f-3f4f2c7a1d55
37
Hh8!qA2vRz9-KpLm4Tn0

Four is the state format version, the schema OpenTofu writes. The long identifier is the lineage, a unique ID minted when the state was first created; OpenTofu checks it before it writes, so it can shout if somebody swapped a different state file in underneath you. Then thirty-seven resources, and one plaintext password sitting there in the open. Marking a value sensitive = true hides it from terminal output. It does not hide it from state, and it never has. So the file you are about to upload is two things at once: a credential store, and a labelled map of your environment. The bucket you put it in inherits both. Read access to state is read access to the secrets inside it. Design the permissions from that assumption.

Point OpenTofu at a Backend

A backend is the shared safe. You declare it inside the terraform block, and yes, the block is still called terraform in OpenTofu, kept that way on purpose so existing Terraform code drops in unchanged. Terraform demands literal strings in there, because it reads that block before it evaluates anything else. OpenTofu 1.8 and newer adds an early evaluation pass that lets you reference input variables and locals in a backend block, but the value has to be resolvable the instant you run init, which means threading it through every command. For per-environment values there is a calmer option that works everywhere: partial configuration. Keep the shape in version control, feed the rest at init time, either as key=value pairs or in a .tfbackend file. The example below uses the S3 backend (Amazon Simple Storage Service, AWS object storage: buckets full of files addressed by a key).

terraform {
  required_version = ">= 1.10.0"
 
  # Partial config: the shape lives in git, the values arrive at init time.
  backend "s3" {
    encrypt      = true   # server-side encryption on every write
    use_lockfile = true   # native S3 locking (OpenTofu 1.10 and newer)
  }
}

bucket     = "acme-tofu-state-prod"
key        = "prod/network/tofu.tfstate"
region     = "us-east-1"
kms_key_id = "arn:aws:kms:us-east-1:111122223333:key/2b9d6c1e-4f77-4a3e-b0f2-9c5d8a1e3b64"

One module, one .tfbackend file per environment. No code edits to aim at a different bucket or prefix. That kms_key_id names a key in KMS (Key Management Service, the AWS service that holds encryption keys and records every use of them), so the encryption at rest sits on a key you own, can audit, and can revoke. Now run init and tell OpenTofu to carry the existing local state up with it.

tofu init -backend-config=envs/prod.tfbackend -migrate-state

Initializing the backend...
Do you want to copy existing state to the new backend?
  Pre-existing state was found while migrating the previous "local" backend to the
  newly configured "s3" backend. No existing state was found in the newly
  configured "s3" backend. Do you want to copy this state to the new "s3"
  backend? Enter "yes" to copy and "no" to start with an empty state.
 
  Enter a value: yes
 
 
Successfully configured the backend "s3"! OpenTofu will automatically
use this backend unless the backend configuration changes.
 
Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Using previously-installed hashicorp/aws v6.14.1
 
OpenTofu has been successfully initialized!

That prompt is the entire migration. Answer yes and OpenTofu uploads the local state to the bucket, then records which backend this directory is bound to in .terraform/terraform.tfstate. Despite the name, that file is a pointer, not your state. It holds the backend type and the settings that got resolved at init, including anything you passed with -backend-config. It is also the first thing to check when a pipeline starts behaving as if it is looking at somebody else's infrastructure.

# which backend is this working directory actually bound to?
jq '{type: .backend.type, config: (.backend.config | {bucket, key, region})}' \
  .terraform/terraform.tfstate

{
  "type": "s3",
  "config": {
    "bucket": "acme-tofu-state-prod",
    "key": "prod/network/tofu.tfstate",
    "region": "us-east-1"
  }
}

One Pen, Chained to the Ledger

A lock is that pen. Before OpenTofu writes state it makes a claim, holds it for the length of the operation, and drops it when it finishes. While the claim is held, any other run that needs to write stops and tells you who has it instead of racing you. With use_lockfile = true the claim is an ordinary object sitting next to the state file, prod/network/tofu.tfstate.tflock. OpenTofu creates it with a conditional write, which is S3's way of saying create this object only if it does not already exist yet. Two runners hitting the bucket in the same millisecond cannot both win. The loser gets HTTP 412 back (Precondition Failed, the response code a web server returns when the condition you attached to your request did not hold) and stops.

tofu apply

╷
│ Error: Error acquiring the state lock
│
│ Error message: operation error S3: PutObject, https response error StatusCode: 412,
│ RequestID: QN4XT7B9C2K1PZ0R, api error PreconditionFailed: At least one of the
│ pre-conditions you specified did not hold
│ Lock Info:
│   ID:        6d1b8f4e-2c30-4d2a-9a1f-8b7e0c5d3a21
│   Path:      acme-tofu-state-prod/prod/network/tofu.tfstate.tflock
│   Operation: OperationTypeApply
│   Who:       runner@gitlab-runner-07
│   Version:   1.10.3
│   Created:   2026-07-21 09:14:22.118374 +0000 UTC
│   Info:
│
│
│ OpenTofu acquires a state lock to protect the state from being written by
│ multiple users at the same time. Please resolve the issue above and try again.
│ For most commands, you can disable locking with the "-lock=false" flag, but this
│ is not recommended.
╵

Read that block like a police report. Who names the user and host holding the claim. Created tells you how long they have had it. Operation tells you whether this is a plan you can wait out or an apply that is halfway through changing real things. In CI (continuous integration, the automated pipeline that runs your builds and applies for you), add -lock-timeout=10m so a queued job waits politely instead of failing the build. Do not reach for -lock=false. It does not clear the lock. It skips locking altogether, which is precisely the concurrent-write accident the backend exists to prevent. Older stacks kept the claim as a row in a DynamoDB (a managed key-value database from AWS) table named by dynamodb_table. That argument still works and is now deprecated in favour of the lock file, and you can set both while you move across, in which case OpenTofu takes both locks.

Sometimes the pen never comes back. A runner gets killed mid-apply, the process dies before it can delete the lock object, and every later run is blocked by a ghost. Read the lock, confirm with the named human that nothing is still running, then release it with the ID from the error. Use force-unlock rather than deleting the object by hand: it checks the ID you pass against the one in the lock, so you cannot accidentally kick out a live apply.

# who is holding it, and since when?
aws s3 cp s3://acme-tofu-state-prod/prod/network/tofu.tfstate.tflock - \
  | jq '{ID, Who, Created, Operation}'
 
# only after confirming that apply is really dead
tofu force-unlock 6d1b8f4e-2c30-4d2a-9a1f-8b7e0c5d3a21

{
  "ID": "6d1b8f4e-2c30-4d2a-9a1f-8b7e0c5d3a21",
  "Who": "runner@gitlab-runner-07",
  "Created": "2026-07-21T09:14:22.118374Z",
  "Operation": "OperationTypeApply"
}
 
Do you really want to force-unlock?
  OpenTofu will remove the lock on the remote state.
  This will allow local OpenTofu commands to modify this state, even though it
  may still be in use. Only 'yes' will be accepted to confirm.
 
  Enter a value: yes
 
OpenTofu state has been successfully unlocked!
 
The state has now been unlocked, and OpenTofu commands should now be able to
obtain a new lock on the remote state.

Harden the Bucket, Then Prove It

That bucket is now the most valuable object in the account, so think about it the way an attacker would. Read access hands them your secrets and a labelled map of every subnet, role, and security group you run. Write access is worse, and much quieter. Change the ID recorded against a resource and the next apply manages something the attacker owns instead. Delete an entry and OpenTofu forgets the resource exists, leaving it running with nobody watching it while a fresh copy gets built beside it. None of that touches your .tf files, so no code review will ever catch it. Three bucket settings decide whether a stolen credential is a bad afternoon or a breach.

aws s3api get-bucket-versioning   --bucket acme-tofu-state-prod
aws s3api get-public-access-block --bucket acme-tofu-state-prod
aws s3api get-bucket-encryption   --bucket acme-tofu-state-prod

{
    "Status": "Enabled"
}
{
    "PublicAccessBlockConfiguration": {
        "BlockPublicAcls": true,
        "IgnorePublicAcls": true,
        "BlockPublicPolicy": true,
        "RestrictPublicBuckets": true
    }
}
{
    "ServerSideEncryptionConfiguration": {
        "Rules": [
            {
                "ApplyServerSideEncryptionByDefault": {
                    "SSEAlgorithm": "aws:kms",
                    "KMSMasterKeyID": "arn:aws:kms:us-east-1:111122223333:key/2b9d6c1e-4f77-4a3e-b0f2-9c5d8a1e3b64"
                },
                "BucketKeyEnabled": true
            }
        ]
    }
}

Versioning is your undo button. Every apply writes a new version of the object and keeps the previous one, so a corrupted write or a mistaken tofu state rm becomes a restore instead of an incident. The public access block is the boarded-up window: four switches that stop anyone from making the bucket readable by the whole internet, even by accident. Default encryption with your own KMS key turns every read into a permission you granted and an event you can see. Versioning is also how you answer the question that comes up in every post-incident review: what did this look like before Tuesday?

aws s3api list-object-versions --bucket acme-tofu-state-prod \
  --prefix prod/network/tofu.tfstate \
  --query "Versions[?Key=='prod/network/tofu.tfstate'].[LastModified,Size,VersionId]" \
  --output text | head -3

2026-07-21T09:11:48+00:00	48217	8kA1n0mQ7bT2sVx4hLd9YpRc3EwZfJu6
2026-07-20T17:02:05+00:00	48190	QpZ3rY7xN1cB6vK0mHe5tJw8dLsA2fUg
2026-07-19T11:44:31+00:00	47903	Rt4bC9nW2qE7yU1oI6pX3aZ5vM8kS0hD

Then cut the permissions down to the shape of the job. Think of it as a key cut for one drawer rather than the master key to the building. A pipeline that owns the network stack needs to read and write exactly one object, take and release exactly one lock, and use exactly one encryption key. It has no business listing the whole bucket, and it never needs to delete the state file. Below is that idea written as an IAM (Identity and Access Management, the AWS service that decides which principal may call which API) policy. Each Resource line is an ARN (Amazon Resource Name, the unique address AWS gives every object).

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListOnlyThisPrefix",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::acme-tofu-state-prod",
      "Condition": { "StringLike": { "s3:prefix": "prod/network/*" } }
    },
    {
      "Sid": "ReadWriteOneStateFile",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Resource": "arn:aws:s3:::acme-tofu-state-prod/prod/network/tofu.tfstate"
    },
    {
      "Sid": "TakeAndReleaseItsLock",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
      "Resource": "arn:aws:s3:::acme-tofu-state-prod/prod/network/tofu.tfstate.tflock"
    },
    {
      "Sid": "UseTheStateKey",
      "Effect": "Allow",
      "Action": ["kms:Encrypt", "kms:Decrypt", "kms:GenerateDataKey"],
      "Resource": "arn:aws:kms:us-east-1:111122223333:key/2b9d6c1e-4f77-4a3e-b0f2-9c5d8a1e3b64"
    }
  ]
}

One caveat on that first statement. The prefix condition is deliberately tight, and OpenTofu workspaces list under a separate env:/ prefix, so if this stack uses workspaces you have to widen that one condition or tofu workspace list fails. Now notice what is absent. There is no s3:DeleteObject on the state object, so a stolen CI token can overwrite your ledger but cannot erase it, and versioning makes every overwrite recoverable.

For detection, switch on CloudTrail data events for that bucket. CloudTrail (the AWS audit log that records who called which API, and when) captures bucket-level changes by default but not object reads, so without data events enabled a GetObject against your state leaves no trace anywhere. That is a camera pointed straight at the vault door with the power off. An advanced event selector matching eventCategory of Data, resources.type of AWS::S3::Object, and resources.ARN starting with the bucket ARN gives you one record per read and per write, with the calling principal attached. The alert then writes itself. The CI role touches this object on a rhythm you recognise, so a human IAM user pulling it at 02:00 on a Sunday is what you page on.

Share Outputs, Do Not Copy Them

Teams split infrastructure across several state files, network in one, databases in another, so one bad apply cannot take out everything at once. Then one stack needs a value another stack produced. Hardcoding a subnet ID works right up until somebody rebuilds the network. Reading it live is better: the terraform_remote_state data source fetches another stack's state read-only and hands you its root-level output values.

data "terraform_remote_state" "network" {
  backend = "s3"
 
  config = {
    bucket = "acme-tofu-state-prod"
    key    = "prod/network/tofu.tfstate"
    region = "us-east-1"
  }
}
 
resource "aws_instance" "app" {
  ami           = "ami-0c0b74d29acc0f2a2"
  instance_type = "t3.micro"
 
  # pull what the network stack published, no hardcoded IDs
  subnet_id = data.terraform_remote_state.network.outputs.private_subnet_id
}

Know what that costs before you spread it around the company. There is no server-side filter here. The data source downloads the whole state file and picks the outputs out on your own machine, so granting a team read access to one output grants them read access to every resource attribute and every secret in that state. The fetched values also get written into your plan file, which means plan artifacts in a pipeline need the same protection as state itself. Keep terraform_remote_state inside a single trust boundary, between stacks the same team owns. Crossing a boundary, publish the value to Parameter Store (AWS Systems Manager Parameter Store, a small key-value store with permissions of its own) and read it back with the aws_ssm_parameter data source, so consumers get that one value and nothing else.

Then go look at the buckets you already own. For each one, list the principals holding s3:GetObject on the state prefix and read that list out loud. If it includes a broad read-only auditor role, or a group named after an entire department, those people are holding your database passwords right now, and no amount of sensitive = true in the code takes that back.

Try this

Run jq -r '.version, .lineage, (.resources | length)' terraform.tfstate on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: the old local state does not delete itself. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related