Terraform interview questions

The short version is: state is Terraform's memory. It's a JSON map from config addresses to real resource IDs and attributes, so Terraform can diff, own resources, and know what to update or destroy.

terraform state list
terraform state show aws_instance.web
# state = address -> provider ID + attributes

I'd describe it as write → init → plan → apply. You write HCL, init pulls providers and wires the backend, plan shows the diff, apply makes it real. Destroy is when you're tearing the stack down.

terraform init
terraform plan -out tf.plan
terraform apply tf.plan

Plan figures out the create/update/replace/destroy actions; apply actually runs them. If I save a plan with -out and apply that file, I'm running exactly what we reviewed — not a surprise second plan.

terraform plan -out tf.plan
terraform show tf.plan
terraform apply tf.plan

Because local state doesn't scale for a team. Remote state is one source of truth, and locking — the S3 lockfile, HCP Terraform, whatever your backend provides — stops two applies from writing at once and corrupting state. On the S3 backend that means use_lockfile = true now; the DynamoDB table is the deprecated way.

terraform {
  backend "s3" {
    bucket       = "tf-state"
    key          = "prod/network.tfstate"
    region       = "us-east-1"
    use_lockfile = true   # S3-native lock (Terraform 1.11+); dynamodb_table is deprecated
    encrypt      = true
  }
}

Init installs providers and modules, configures the backend, and writes the lockfile. I re-run it after backend or provider changes. It doesn't change infrastructure by itself.

terraform init
terraform init -upgrade   # bump within constraints
# commit .terraform.lock.hcl

One stack writes outputs into its state; another reads them with terraform_remote_state — or better, a published module contract. I'd rather consume explicit outputs than dig through another team's full state file.

data "terraform_remote_state" "net" {
  backend = "s3"
  config = {
    bucket = "tf-state"
    key    = "prod/network.tfstate"
    region = "us-east-1"
  }
}
# data.terraform_remote_state.net.outputs.vpc_id

It reconciles state with what's actually in the cloud without applying config changes. The standalone terraform refresh command is deprecated because it writes state with no review; it equals apply -refresh-only -auto-approve. plan -refresh-only is how I inspect drift; apply -refresh-only accepts reality into state when that's intentional.

terraform plan -refresh-only
terraform apply -refresh-only

fmt rewrites HCL to the canonical style; validate checks syntax and internal consistency — references, types — without talking to the cloud. I run both in CI before plan.

terraform fmt -check -recursive
terraform init -backend=false
terraform validate

I constrain versions in required_providers and commit .terraform.lock.hcl so every laptop and CI run uses the same provider binaries and checksums. Bumps happen on purpose with init -upgrade.

terraform {
  required_version = ">= 1.5"
  required_providers {
    aws = { source = "hashicorp/aws", version = "~> 6.0" }
  }
}

It's a reusable package of resources with inputs and outputs. The root module calls child modules so common patterns — VPC, RDS, IAM — stay DRY and versioned instead of copy-pasted.

module "vpc" {
  source = "./modules/vpc"
  cidr   = "10.0.0.0/16"
}
# module.vpc.subnet_ids

I'd break it down like this: variables are what callers pass in, locals are named expressions computed once inside the module, outputs are what you expose back out or to remote state. Mark secrets sensitive so they get redacted in the UI — they still land in state though.

variable "env" { type = string }
locals { name = "${var.env}-api" }
output "url" { value = aws_lb.web.dns_name }

The short version: count makes N indexed copies; for_each keys instances by a map or set. I prefer for_each whenever identity matters — yanking one item from a count list reshuffles indexes and forces unwanted recreates.

count addresses look like aws_instance.web[1]. Drop index 0 and everything after renumbers, so Terraform plans destroy/create for survivors that didn't really change. for_each keeps stable keys via each.key (a name, AZ, or subnet id), so add/remove only touches that instance. I'd only use count for identical disposable copies where a reshuffle is fine. To move an existing count resource without recreating anything, switch it to for_each, add a moved block from each surviving index to its new key, and plan until it shows moves, not destroys. terraform state mv, one address at a time, is the CLI fallback.

resource "aws_iam_user" "u" {
  for_each = toset(["alice", "bob"])
  name     = each.key
}
# migrating from count: one moved block per surviving index
moved {
  from = aws_iam_user.u[0]
  to   = aws_iam_user.u["alice"]
}

Interviewer often follows with: How do you move an existing count resource to for_each without recreating anything?

A resource creates and manages something; a data source only reads existing infra or provider data at plan time. Data sources never create or destroy.

data "aws_ami" "ubuntu" {
  most_recent = true
  owners      = ["099720109477"]
}
# data.aws_ami.ubuntu.id

Workspaces switch state inside one backend namespace; separate directories — or Terragrunt stacks — with different backend keys isolate blast radius. For prod vs non-prod I'd rather have separate state keys than lean on workspaces alone.

Workspaces are handy for identical configs with different state, but they share the same code and backend config, which tempts applying to the wrong workspace and couples blast radius. Directory-per-env with distinct backend keys makes the environment obvious in the path, lets IAM differ per key, and matches how most teams run CI — one pipeline per stack. I use workspaces sparingly for true clones, not as the primary multi-env strategy.

# envs/prod/network — key = "prod/network.tfstate"
# envs/dev/network  — key = "dev/network.tfstate"
terraform workspace list   # fine for soft clones, not hard isolation

Interviewer often follows with: What happens if someone runs terraform workspace select prod on a laptop with broad credentials?

for expressions transform collections; dynamic blocks generate repeated nested blocks — ingress rules, tags — from a list so the module stays DRY when the length varies.

dynamic "ingress" {
  for_each = var.ports
  content {
    from_port = ingress.value
    to_port   = ingress.value
    protocol  = "tcp"
    cidr_blocks = ["10.0.0.0/8"]
  }
}

Provisioners run imperative scripts Terraform can't really model in the plan or track in state, so they break idempotency and drift detection. I'd rather bake an AMI, use cloud-init, or hand off to config management.

local-exec and remote-exec mostly fire on create/destroy, hide side effects from the dependency graph, and fail differently across CI runners. I'd bake AMIs, use user_data/cloud-init, or Ansible rather than make SSH-from-Terraform the default path.

I'd publish to a private registry or git tags with semver, pin callers to something like ~> major.minor, and keep a changelog. Breaking input renames get a deprecation window or a new major so consumers upgrade on purpose.

Without versioning, every consumer floats on main and breaks together. Registry modules — or git ref=v1.4.2 — plus required_version and required_providers give reproducible installs. Contract-wise I want stable outputs, documented variables, examples/, and CI that runs terraform test or plan against a fixture. For a breaking change I'd bump major, keep a v1 branch for a while, and migrate stacks via PR. I never point production at latest git main as the module source.

module "vpc" {
  source  = "app.terraform.io/acme/vpc/aws"
  version = "~> 2.3"
}

Interviewer often follows with: How do you test a module before you publish a new minor?

I write .tftest.hcl files with run blocks and assert conditions. command = plan runs are fast logic checks; the default apply mode creates real resources and Terraform tries to destroy them at the end of the file. Mock providers (1.7+) let tests run without cloud credentials.

terraform test ships with Terraform 1.6+, and each run block plans or applies the module with its own variables. Plan-mode runs with mock_provider or override_resource are cheap enough for every PR; apply-mode runs in a sandbox account catch provider-side errors a plan never sees, so I run those on merge or nightly. Cleanup is best effort, so the sandbox still needs its own reaper. Tests prove the module behaves as designed; policy checks on plan JSON still decide what is allowed in production.

# tests/vpc.tftest.hcl
mock_provider "aws" {}

run "cidr_is_applied" {
  command = plan
  variables {
    cidr = "10.0.0.0/16"
  }
  assert {
    condition     = aws_vpc.this.cidr_block == "10.0.0.0/16"
    error_message = "VPC CIDR does not match the input"
  }
}

# terraform test

Interviewer often follows with: Which assertions belong in plan-mode tests, and which need a real apply?

It creates the replacement before destroying the old one, so you cut downtime for things like instances behind a load balancer. prevent_destroy and ignore_changes are the other lifecycle knobs I'd reach for.

resource "aws_instance" "web" {
  # ...
  lifecycle {
    create_before_destroy = true
    ignore_changes        = [tags["LastTouched"]]
  }
}

Drift is when the real cloud diverges from state — usually someone clicked in the console. terraform plan or plan -refresh-only shows it; then I either revert the change or codify it, and tighten IAM so it doesn't keep happening.

terraform plan -refresh-only
terraform apply -refresh-only   # accept reality into state

I prefer an import block over the older terraform import command: plan previews the import, apply writes it to state, and plan -generate-config-out (experimental) can draft the resource block. If the config matches reality, the plan shows only the import, with nothing to add, change, or destroy.

import {
  to = aws_s3_bucket.logs
  id = "my-logs-bucket"
}
# terraform plan   # expect: Plan: 1 to import, 0 to add, 0 to change, 0 to destroy.
# terraform apply  # the import lands in state on apply

I'd add a moved block — or run state mv — so the address updates in state without touching the cloud object. After that, plan should show a move, not destroy/create.

Terraform matches by address. Renaming aws_instance.web to module.app.aws_instance.web looks like delete-old plus create-new unless you tell it they're the same object. moved blocks are reviewable in PRs and work in CI; state mv is the CLI equivalent for one-offs. After the move I verify with plan that only the address changed. Pair with import when you're bringing brownfield into a module for the first time.

moved {
  from = aws_instance.web
  to   = module.app.aws_instance.web
}
terraform plan   # should show move, not replace

Interviewer often follows with: When would you reach for removed blocks instead?

I'd only use it as an incident escape hatch to apply one resource. Routine -target applies a partial graph and can leave dependent state inconsistent — usually a sign the modules are too coupled.

-target still refreshes and plans the dependency closure of the target, but it skips siblings, so outputs and cross-resource contracts can drift from what a full apply would do. I prefer smaller state files per component so a normal plan is reviewable. If I must target in an incident, I follow with a full plan in the same change window to confirm nothing else is pending.

terraform apply -replace=aws_instance.web
# prefer -replace over legacy terraform taint
# avoid: terraform apply -target=... in routine CI

Interviewer often follows with: How would you split a 2k-resource state so targeting isn't needed?

The short version: it parses config, builds a graph from references — and depends_on — refreshes current state, then walks that graph to emit create/update/replace/destroy. Parallelism and order come from the graph, not file order.

Implicit edges come from interpolating attributes, like subnet_id = aws_subnet.a.id. Explicit depends_on is for hidden side effects. Cycles fail at plan. Replace vs update depends on ForceNew attributes in the provider schema. That's why count index reshuffles recreate, why outputs wait on resources, and why -target is dangerous. terraform graph can visualize it; in interviews I walk parse → provider schemas → graph → refresh → diff → plan file.

terraform graph | dot -Tpng > graph.png
terraform plan -out tf.plan
terraform show -json tf.plan | jq '.resource_changes[].change.actions'

Interviewer often follows with: How do provider ForceNew attributes show up in a plan?

terraform apply -replace=ADDRESS recreates that resource — the modern replacement for taint. I'd use it sparingly and be clear on destroy order and data loss first.

terraform apply -replace=aws_instance.web

I'd read the error and the state — some resources may be created, others pending. Fix the config or cloud issue, re-plan, and apply again; Terraform continues from state. I don't manually delete objects without updating state.

Partial applies are normal under API rate limits or IAM errors. State should reflect what succeeded; the next plan shows remaining creates or tainted replacements. If the real object exists but state missed it — rare crash after create — I import. If state has an object that failed provisioning, I replace carefully. I never hand-edit state JSON. After recovery, a full plan should be empty. Post-incident I'd add retries, smaller blast radius, and pre-apply IAM checks.

terraform plan -out tf.plan
terraform apply tf.plan
# if object exists but missing from state:
# terraform import aws_s3_bucket.logs my-logs-bucket

Interviewer often follows with: How do you practice disaster recovery for the state bucket itself?

destroy deletes real infrastructure and removes it from state. state rm only forgets the object in state — the cloud resource keeps running until you delete it another way. A removed block with lifecycle { destroy = false } (1.7+) does the same through a reviewed plan.

terraform state rm aws_instance.legacy
# instance still exists in AWS — manage or delete out of band

I'd stop, verify which attribute forces new, check for data loss and outage, and either stage a blue/green migration, use create_before_destroy where it's safe, or revert the config — I never apply a surprise database replace on prod.

Provider schemas mark ForceNew on things like engine version jumps, AZ, or storage encryption toggles. Plan shows -/+ replace. For stateful data stores, replace usually means destroy then create unless create_before_destroy and a cutover plan exist. I'd terraform show the plan JSON for that resource, read the provider docs, take a snapshot, then decide migrate vs undo. Prefer in-place updates or a separate blue instance plus cutover. Document in the PR why replace is acceptable. Unexpected replace on data is an incident gate, not a green apply.

terraform show -json tf.plan | jq '.resource_changes[]
  | select(.address|test("aws_db_instance"))
  | {address, actions:.change.actions, before:.change.before, after:.change.after}'
# if ForceNew surprise: revert PR or design blue/green cutover

Interviewer often follows with: Which RDS changes are in-place versus ForceNew in the AWS provider?

I'd pin and upgrade in a dedicated branch, read the upgrade guide, run plans in dev first, use moved blocks or intermediate module versions, and only promote when the plan noise is understood — never big-bang prod on a Friday.

Major provider bumps often rename resources — think the aws_s3_bucket_* split in AWS provider 4.0 — or change ForceNew. My path: changelog and upgrade guide → update required_providers → init -upgrade in a sandbox → plan and triage every destroy → apply moved blocks or state replace-provider if needed → terraform test / CI against fixtures → promote env-by-env. Keep the lockfile as the promotion artifact. If destroys are unavoidable, schedule maintenance and backups. Rollback is revert lockfile plus provider constraint and re-init. Treat provider majors like app majors with canaries.

# required_providers aws = { version = "~> 6.0" }
terraform init -upgrade
terraform providers lock -platform=linux_amd64
terraform plan -out tf.plan   # triage every destroy

Interviewer often follows with: How do you keep app teams unblocked while a platform provider upgrade is in flight?

Plan on every PR and post it for review; apply only on merge to a protected branch from a trusted pipeline. I'd use OIDC for short-lived cloud creds, remote state with locking, and apply the saved plan artifact — not a fresh plan on main.

Anti-patterns I push back on: apply from laptops to prod, static AWS keys in CI, apply without the reviewed plan file. The pattern that works: fmt/validate/tflint/checkov on PR → plan -out → upload the plan artifact → policy on plan JSON → approval → apply that artifact on main. Branch protection and environment approvals stop fork PRs from applying. Pin providers via a committed lockfile.

terraform plan -out=tf.plan
terraform show -json tf.plan > plan.json
# on main after approval:
terraform apply -input=false tf.plan

Interviewer often follows with: How do you stop a compromised PR from reading production state?

I'd prefer OIDC or workload identity: the pipeline gets a short-lived token scoped to repo and branch, exchanged for cloud credentials. No long-lived access keys sitting in the secret store.

GitHub Actions or GitLab OIDC into cloud STS with a trust policy conditioned on sub — repo — ref — branch — and optionally the workflow. IAM role allows only the APIs that stack needs; network stack ≠ app stack. Locally, developers use SSO profiles, not shared keys. HCP Terraform and Terraform Enterprise can do dynamic provider credentials. Falling back to static keys means rotation pain, leak risk, and harder attribution in CloudTrail.

# CI sets AWS_ROLE_ARN + web identity token file
provider "aws" {
  region = "us-east-1"
  # credentials from env / OIDC — no static keys in code
}

Interviewer often follows with: How do you scope the trust policy so only main can deploy prod?

I run static scanners like Trivy (the successor to tfsec) or checkov on the code, and OPA/Conftest or Sentinel on the plan JSON in CI, so public buckets and open security groups fail before merge.

terraform show -json tf.plan > plan.json
conftest test plan.json -p policy/
checkov -f plan.json

I'd ignore .terraform/, crash logs, *.tfstate*, secret tfvars, and plan binaries. Commit the .tf files, modules, lockfile, and example tfvars without secrets.

.terraform/
*.tfstate
*.tfstate.*
crash.log
*.tfplan
tf.plan
secret.auto.tfvars

I'd rotate the password immediately, purge it from git history if it was pushed, revoke anything derived from it, add secret scanning to CI, and move secrets to a manager injected at apply time — deleting the file alone isn't enough.

Deleting the commit from main doesn't remove clones or CI caches. Rotate first. Use git filter-repo or BFG only with a coordinated force-push policy, then invalidate leaked credentials. Prevention: pre-commit scanners like gitleaks, blocked patterns, and ephemeral secrets from Vault or SSM. sensitive variables still land in state — treat state access like secret access.

# 1) rotate the DB password in the engine
# 2) gitleaks git -v .   # full history; detect is deprecated since v8.19
# 3) remove *.auto.tfvars secrets from repo; use CI OIDC + Vault

Interviewer often follows with: Does git rm of the tfvars file remove it from history?

I'd treat it as a control gap: either the plan JSON didn't include the public ACL/policy change, the rule was too narrow, or we were missing a post-apply AWS guardrail like account Block Public Access — then fix detection and add a preventive account control.

Plan-time policy sees intended Terraform changes, not console clicks or a separate stack. Common gaps: resource created private then a public_access_block omitted; fixture used count=0; rule checked ACL but not policy JSON; apply used a different var-file than CI. Hardening: test policies against real plan fixtures that include public ACL and policy docs, require aws_s3_bucket_public_access_block, enable account-level BPA, and add a post-apply control that pages on Public. Defense in depth beyond a green Conftest run.

# conftest: deny if change after_* has ACL public-read or Policy Principal=*
# plus account BPA:
# aws s3control put-public-access-block --account-id ... --public-access-block-configuration ...

Interviewer often follows with: How do you regression-test the OPA policy so this class of miss can't come back?

I'd halt automation, assess blast radius from state and plan logs, restore from state history or rebuild, then fix workspace/backend selection so prod applies can't be selected by a mislabeled PR comment.

Usual root causes: shared backend with workspace select driven by free text, missing environment protection, or directory allowlists that are too broad. Recovery: figure out what changed from Atlantis logs and state versions, state pull a previous version if versioning is on, or import/recreate. Prevention: one backend key per env — no workspace roulette — Atlantis project bindings by path, CODEOWNERS plus approvals for prod paths, OIDC roles that can't assume prod from non-prod dirs, and plan-only on forks. I want both incident recovery and structural elimination of workspace ambiguity.

# atlantis.yaml — explicit dir → workspace/backend
projects:
  - name: prod-network
    dir: envs/prod/network
    autoplan: { when_modified: ["**/*.tf"] }
# IAM: role only assumable when claim path matches prod

Interviewer often follows with: Why is directory-per-env safer than terraform workspace select in chatops?

I split by blast radius: per environment and per component — network, data, app — each with its own backend key. Share logic via versioned modules; cross-stack links via outputs or remote state.

One giant state means one lock, one blast radius, and multi-hour plans. Split keys so a bad app apply can't destroy the VPC. I layer network → platform → app with clear ownership, map each directory to a CI pipeline, and document which stack may read whose remote state so we don't create secret circular dependencies.

# prod/network.tfstate
# prod/data.tfstate
# prod/app.tfstate

Interviewer often follows with: How do you handle a shared platform stack that many apps depend on?

HashiCorp moved Terraform from MPL 2.0 to the Business Source License (BUSL 1.1) in August 2023; OpenTofu is the Linux Foundation fork of the last MPL code. The licence targets competing commercial offerings, not normal internal use, so I choose on features, support, and what the team already runs.

OpenTofu works with state written by Terraform up to 1.5.x and uses the same providers through its own registry, but the two have diverged since: OpenTofu added client-side state and plan encryption, and Terraform added HCP-only features such as Stacks. I pick one binary per repo, pin it in CI, and never let laptops and pipelines mix the two against the same state.

terraform {
  encryption {
    key_provider "pbkdf2" "main" {
      passphrase = var.state_passphrase
    }
    method "aes_gcm" "main" {
      keys = key_provider.pbkdf2.main
    }
    state {
      method = method.aes_gcm.main
    }
    plan {
      method = method.aes_gcm.main
    }
  }
}

Interviewer often follows with: What would you check before pointing tofu at state last written by a recent Terraform release?

Keep one module; pass variables via tfvars files, CI env, or env-specific auto.tfvars. I'd rather do directory-per-env with shared modules than copy-paste resource blocks.

terraform plan -var-file=env/prod.tfvars -out tf.plan
# env/prod.tfvars: instance_type = "m5.large"

I'd invent new state keys, use state mv or moved blocks in coordinated PRs, keep one writer at a time, and prove empty plans on both old and new stacks before deleting the monolith key — never copy state JSON by hand mid-apply.

Split by blast radius — network, data, app. Sequence I'd follow: freeze applies on the monolith, create empty backends for the new keys, state pull the monolith, state mv resources into the new state, remove resources from old config via removed/moved, plan both sides until there are no unexpected destroys, then cut CI over to per-component pipelines. A config-driven alternative (1.7+) is removed blocks with destroy = false in the monolith plus import blocks in the new stacks, so every step shows in a reviewed plan; terraformer/import help when addresses never matched. Risks are dual ownership, orphaned resources, and lock contention. Locking strategy, a monolith snapshot for rollback, and proving no destroy of prod data are the expert bar.

terraform state mv -state=old.tfstate -state-out=network.tfstate   aws_vpc.main aws_vpc.main
# then commit config that only the network stack owns aws_vpc.main

Interviewer often follows with: How do you handle outputs that other stacks still read via remote state during the split?

I'd give each region its own state key and module instantiation, encode promotion and failover outside a single apply, and never point one state at resources in both regions with overlapping lifecycle.

Patterns that work: identical module called twice with provider aliases and separate state or roots; primary Terraform plus DR bootstrapped from AMIs/snapshots with a runbook; or active-active with explicit data replication resources. Avoid one resource address that can destroy both. Prefer region-coded backend keys. Failover is usually DNS/traffic plus data replication health — not terraform destroy on primary. Test DR in non-prod regularly. What matters is isolated state and an honest boundary between IaC and data-plane DR.

provider "aws" {
  alias  = "primary"
  region = "us-east-1"
}
provider "aws" {
  alias  = "dr"
  region = "us-west-2"
}
module "vpc_primary" {
  source    = "./modules/vpc"
  providers = { aws = aws.primary }
}
module "vpc_dr" {
  source    = "./modules/vpc"
  providers = { aws = aws.dr }
}

Interviewer often follows with: Would you put Route53 failover records in the primary state or a global DNS stack?

I'd confirm no Terraform process still holds the lock, then force-unlock with the ID and immediately re-plan. If the holder is alive, say a teammate's apply, I wait for it to release and re-plan instead: unlocking over a live apply is how you corrupt state.

Stale locks are common after OOM-killed runners or cancelled pipelines. Check the lock object (the S3 .tflock file or a legacy DynamoDB row) or the HCP Terraform UI for LockID and who created it, inspect CI logs for a still-running job, only then force-unlock. force-unlock is for abandoned locks, not for racing a living apply. After unlock, plan to detect a partial apply — resources created but not in state, or vice versa. Prevention: shorter applies, one writer per state key (split keys so two teams never share a lock), runner shutdown hooks that cancel cleanly, and alerts on lock age. Unlock is an incident step with verification, not a reflex.

terraform force-unlock <LOCK_ID>
terraform plan -out tf.plan
# if partial apply: state list vs cloud inventory; import or rm as needed

Interviewer often follows with: What evidence proves the lock holder is dead before force-unlock?

Rotate every exposed secret, restrict state ACL and KMS, purge old state versions, and stop putting secret values in attributes Terraform must store: use secret references, ephemeral resources (Terraform 1.10+) for reads, and write-only arguments (1.11+) for writes. sensitive = true only redacts the UI; the value still lives in state.

State stores resource attributes, including sensitive ones, in JSON: any random_password, provider-returned secret, data source read, or non-ephemeral variable lands there in plaintext. Even with sensitive = true on outputs, state and plan files can still hold the value. Containment: rotate DB passwords and tokens, enable bucket encryption plus IAM least privilege, delete historical state object versions that contain the secret, and audit who could read the bucket. Redesign: write secrets to Vault or AWS SM via a separate process and pass ARNs or names into Terraform, read them through ephemeral resources, and set them with write-only arguments where the provider supports them. Prefer dynamic secrets over long-lived DB passwords in managed resources. OpenTofu 1.7+ can also encrypt state and plan files client-side; Terraform relies on backend encryption. Apply from CI with least-privilege OIDC roles and short-lived credentials. Never commit state; never share plan files with secrets in chat.

# rotate the DB password in the provider/console first
# then remove password from .tf — reference aws_secretsmanager_secret by ARN
# scrub: delete prior s3://tf-state/... object versions after confirming backups

Interviewer often follows with: Does marking an output sensitive=true remove the value from state?

Halt all applies, reconstruct what was destroyed from state history and cloud audit trails, restore data from backups, then hard-separate backend keys and IAM so prod credentials can't touch staging paths — and vice versa.

Wrong-env destroy is usually workspace/backend mis-selection or a copied -var-file. Recovery depends on resource type: re-apply from last known good state if resources still exist; restore DB snapshots for data; import after manual recreate when needed. Forensics: S3 state versions, CloudTrail, CI job parameters. Prevention: directory-per-env, OIDC role binding by path, protected environments, plan artifacts that show the backend key, and deny destroy on prod without break-glass. Don't rely on resource name prefixes alone.

terraform state pull > /tmp/now.json
aws s3api list-object-versions --bucket tf-state --prefix envs/staging/
# restore prior version to a file, compare, import/recreate carefully

Interviewer often follows with: Why is IAM path-scoped OIDC stronger than telling engineers to double-check -var-file?

I'd treat console edits as unauthorized drift unless there was an incident break-glass. Either revert in AWS by applying Terraform, or intentionally import the change into code — never leave two sources of truth.

I detect drift with plan -refresh-only or regular CI plans. If the console change was emergency access, codify it in a PR quickly or schedule the revert. If it was shadow IT, apply to restore the baseline and page the actor. I avoid lifecycle ignore_changes on security groups unless there's a documented dual-control model — it hides future drift. Longer term: SCPs or Config rules that detect manual SG changes, and make Terraform the only writer via IAM permissions boundaries.

terraform plan -refresh-only
terraform plan -out tf.plan   # will show SG rule churn
# either: apply to enforce code, or PR the console rules into .tf then apply

Interviewer often follows with: When is ignore_changes on an SG rule a reasonable exception?

The pessimistic constraint allowed a new minor that changed ForceNew behavior. I'd pin exact module versions, because the dependency lock file only covers providers, and promote module bumps through environments on purpose.

~> 3.2 allows 3.2.0 through <4.0.0, so a 3.5 module can rewrite resources. .terraform.lock.hcl locks providers, not remote module versions — module pins must be exact in the calling module or managed by a private registry promotion channel. Response: revert the module version pin, re-init, confirm a safe plan, then land 3.5 via a deliberate PR with changelog review. Policy: no floating module versions on prod roots; Dependabot PRs for bumps.

module "vpc" {
  source  = "app.terraform.io/org/vpc/aws"
  version = "3.2.4"   # exact — not ~> 3.2
}
terraform init
terraform plan

Interviewer often follows with: What does the dependency lockfile guarantee for modules vs providers?

I'd inventory cloud vs state, import the missing addresses, and only apply when plan shows zero unwanted creates — never apply "to fix" while duplicates are still pending.

Partial import leaves Terraform thinking un-imported objects should be created, which fails on uniqueness or creates shadow resources. I'd terraform state list, compare to AWS CLI inventory, import each missing address with the correct ID, and plan until clean. For large trees, generate import blocks and apply imports without creating. If a create already leaked, delete the duplicate carefully or remove it from config. Lock the state during the repair window so nobody else applies.

terraform state list | grep subnet
aws ec2 describe-subnets --filters Name=vpc-id,Values=vpc-… \
  --query 'Subnets[].SubnetId'
terraform import aws_subnet.a subnet-abc
terraform plan   # must show no subnet creates

Interviewer often follows with: What risk does applying a plan with create on an existing subnet ID collision create?

Different var-files, backend or workspace, provider versions, or credentials seeing different accounts. I'd diff init lockfiles, -var flags, and terraform version between environments until the plans match.

Common deltas: missing -var-file=prod.tfvars locally, wrong AWS profile or account, stale local state when CI uses remote, provider version drift without committing the lockfile, or CI using -refresh=false vs local refresh. Print terraform version, provider lock hashes, backend config, and the exact plan command in both places. Require CI to upload plan JSON as an artifact. The plan applied to prod must be the plan CI produced — never apply from a laptop against prod state.

terraform version
terraform providers
git status -- .terraform.lock.hcl
# CI and local: same -var-file, same backend key, committed lockfile

Interviewer often follows with: Should prod apply ever use terraform apply without a saved plan file from CI?

Stop applies, restore the previous state object version, fix the migration procedure, and only then re-point the backend — never keep applying against an empty or partial state.

Migrations go wrong when people init -migrate-state against the wrong source, overwrite with local empty state, or copy JSON by hand. Recovery: S3 versioning or HCP Terraform state history to restore the last good blob, verify state list counts, then redo migration with pull → push or the official migrate path. Preflight checklist: resource count before and after, lock during cutover, backup object copied aside. Empty state plus apply is a mass create/destroy disaster.

aws s3api list-object-versions --bucket tf-state --prefix prod/app.tfstate
# get-object prior VersionId to ./recovered.tfstate
terraform state push ./recovered.tfstate   # only after validation

Interviewer often follows with: What safeguard does state push need so you don't overwrite a newer good state?

I'd remove ignore_changes in a controlled PR, expect a replace or refresh plan, and roll instance refresh or blue-green so image updates become intentional again — not a surprise mass replace.

ignore_changes is a sharp tool: it silences drift security cares about — AMI, user_data, SG rules. Unwinding: decide the replace strategy — ASG instance refresh, blue/green, create_before_destroy — take backups, remove the ignore, plan, apply in non-prod first. Document why any remaining ignores exist. Prefer a Packer channel plus explicit var.ami_id bumps over permanent ignore. Trade day-2 console stability against patch compliance explicitly.

# remove: lifecycle { ignore_changes = [ami] }
terraform plan -out tf.plan
# prefer ASG instance_refresh / blue-green over in-place shock

Interviewer often follows with: What's a safer pattern than ignore_changes when ops sometimes rebakes AMIs out of band?

Use moved blocks — or state mv — from old addresses to new ones so Terraform rewires state without touching real resources, then apply the no-op plan.

for_each keys are part of the resource address. Renaming keys looks like delete old plus create new. Terraform 1.1+ moved blocks express the rename in config; older versions use state mv. Always plan and confirm actions are empty or update-only before apply. Do this in a dedicated PR with no other churn. For modules, moved can reference module addresses too.

moved {
  from = aws_subnet.this["public-a"]
  to   = aws_subnet.this["subnet-abc123"]
}
terraform plan   # expect no destroys if mapping is complete

Interviewer often follows with: Can you move addresses across different resource types with moved blocks?

I'd freeze writers, treat cloud as reality for inventory, rebuild state with import and refresh from a backup point, and fix process so force-unlock needs two-person approval.

Split-brain after dual apply is the nightmare locking exists to prevent. Steps: disable CI apply, snapshot current cloud inventory, recover last pre-incident state version, plan -refresh-only to see divergence, then import missing resources, state rm ghosts, and targeted applies to converge. Don't keep applying hoping it heals. Process controls: break-glass unlock runbook, lock-age alerts, one pipeline per key, and preferably HCP Terraform or Terraform Enterprise run queuing. You should be able to narrate forensic order without guessing.

terraform state pull > /tmp/suspect.json
# compare aws inventory vs state list; import orphans; state rm ghosts
terraform plan -refresh-only
# re-enable CI only when plan is understood

Interviewer often follows with: Why is cloud inventory usually safer than state file wins after a dual apply?