What Terraform is & how it works

Declarative, cloud-agnostic provisioning.

Write down what should be in your fridge. Two litres of milk, a dozen eggs, one jar of coffee. Hand that list to someone and they open the fridge, look at what is already in there, and buy only the difference. Give them the same list tomorrow and they do not come home with twelve litres of milk. Terraform is that shopper. You write down the infrastructure that should exist, Terraform reads what actually exists right now, and it makes only the changes needed to close the gap. Nothing else.

Terraform is the best known tool for Infrastructure as Code, usually shortened to IaC, which means describing your servers and networks in text files instead of clicking them into being by hand. Infrastructure is a wide word here. Virtual machines, networks, firewall rules, databases, DNS records (Domain Name System, the internet's address book, which turns names like shop.example.com into numbers), storage buckets, Kubernetes clusters, GitHub repositories, Cloudflare settings, even the on-call rota. Those text files sit in Git next to your application code, so every change turns up as a commit with an author, a timestamp, and, if you require pull requests, somebody who said yes. For security work that history is the whole point. Without it you get 'the firewall rule changed at some point'. With it you get 'Priya widened it on 14 March, here is the pull request, here is who approved it'. One bit of housekeeping before we go on: HashiCorp changed Terraform's licence in 2023, which is why you will also meet OpenTofu, a fork that kept the older open source terms. Same language, same commands, same ideas. Everything below applies to both.

Scripts Give Orders, Terraform States a Result

Most people reach for a shell script first, because a script is obvious. Here is one that builds a security group (a virtual firewall you attach to cloud machines, holding rules for what may come in and what may go out) inside a VPC (Virtual Private Cloud, your own walled-off network inside Amazon Web Services), then opens SSH (Secure Shell, the encrypted remote-login protocol that listens on TCP port 22) to the office and nowhere else. That 203.0.113.10/32 is CIDR notation (Classless Inter-Domain Routing, the slash format for describing ranges of addresses), and /32 means one address exactly.

#!/usr/bin/env bash
set -euo pipefail
 
sg_id=$(aws ec2 create-security-group \
  --group-name bastion-ssh \
  --description "SSH to the bastion from the office only" \
  --vpc-id vpc-0a1b2c3d4e5f60718 \
  --query GroupId --output text)
 
aws ec2 authorize-security-group-ingress \
  --group-id "$sg_id" \
  --protocol tcp --port 22 --cidr 203.0.113.10/32

$ ./provision.sh     # clean account: the group is created, the rule comes back as JSON
$ ./provision.sh     # same script, same intent, one second later

{
    "Return": true,
    "SecurityGroupRules": [
        {
            "SecurityGroupRuleId": "sgr-0c8f1b2a3d4e5f607",
            "GroupId": "sg-0f1e2d3c4b5a69788",
            "GroupOwnerId": "123456789012",
            "IsEgress": false,
            "IpProtocol": "tcp",
            "FromPort": 22,
            "ToPort": 22,
            "CidrIpv4": "203.0.113.10/32"
        }
    ]
}
 
An error occurred (InvalidGroup.Duplicate) when calling the CreateSecurityGroup operation: The security group 'bastion-ssh' already exists for VPC 'vpc-0a1b2c3d4e5f60718'

Before the obvious problem, one detail worth stealing. The script captures the new group's id and passes --group-id on the second call. That is not stylistic. Inside a VPC that is not the account's default, AWS refuses to look a security group up by name, so --group-name either finds nothing or quietly finds a same-named group in the default network. Now the obvious problem. A script is a list of verbs, and every verb assumes the world starts in one exact position. Clean account, it works. Second run, it dies. Point it at an account where somebody already built half of this by hand and it dies somewhere in the middle, leaving a half-finished mess for you to unpick at speed. So you add a check before the create. Then a check before the rule. Then some logic to remove a rule that should not be there any more. Keep going and you have written a worse Terraform, in bash, with no tests. Terraform starts from the other end.

terraform {
  required_version = ">= 1.5"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}
 
provider "aws" {
  region = "eu-west-1"
}
 
resource "aws_security_group" "bastion" {
  name        = "bastion-ssh"
  description = "SSH to the bastion from the office only"
  vpc_id      = "vpc-0a1b2c3d4e5f60718"
 
  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["203.0.113.10/32"]   # the office egress address
  }
 
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

Read that file and notice what is not in it. No create. No update. No order of operations. You named a thing, aws_security_group.bastion, and stated the properties it should have. Whether Terraform creates it, edits it, replaces it or leaves it alone depends entirely on what it finds when it looks. Run it against an empty account and you get one security group. Run it again and you get nothing at all, because nothing needs doing. That property has a name: idempotence. The same input applied over and over leaves the same result, which is what makes it safe to run on a schedule, in a pipeline, at three in the morning, by somebody who has never read the code.

The Three Pictures Terraform Compares

Back to the shopping list. To do the job properly your shopper needs three things: the list (what should be there), the receipt from last time (what they actually bought, which brand, which size), and a look inside the fridge (what is in there this minute). Terraform carries the same three. Your .tf files are the desired state. The state file, normally terraform.tfstate, is Terraform's own record of what it built, including the real cloud identifiers it got back, like sg-0f1e2d3c4b5a69788. A refresh is a live read from the provider's API (Application Programming Interface, the machine-facing control panel a cloud service exposes so that programs can drive it instead of humans clicking). Every plan is those three pictures held up against each other. One caution about that middle picture: state records attribute values in the clear, database passwords and generated keys included, so it belongs in a shared remote backend with access control and locking, such as an S3 bucket with locking switched on, and never in the Git repository.

The graph step is where ordering comes from, and it is the part people find surprising. You cannot ice a cake before you bake it, and Terraform works that out for itself rather than being told. When one resource refers to another, say a subnet reaching for aws_vpc.main.id, Terraform records a dependency edge and assembles a DAG (directed acyclic graph, a set of steps joined by arrows that never loop back on themselves). Then it walks that graph, running independent work side by side, ten resources at a time by default and adjustable with -parallelism=N. You never write 'create the VPC first'. You write the reference, and the ordering falls out of it. Anything that references nothing has no reason to wait, which is why a big first apply finishes faster than you expect.

Terraform Itself Knows Nothing About AWS

The terraform binary is one executable. No agent, no server, nothing running in the background. It understands HCL (HashiCorp Configuration Language, the syntax those .tf files are written in), the dependency graph, and the state format. It has no idea what a security group is. Think of Terraform as an office manager who writes purchase orders in a house style every supplier understands, and providers as the couriers who actually know the shop, the address and the door code. A provider is a separate program that Terraform downloads and runs for you, and every real API call comes out of it, never out of Terraform. hashicorp/aws speaks to Amazon, hashicorp/azurerm to Microsoft Azure, hashicorp/google to Google Cloud, hashicorp/kubernetes to a cluster, cloudflare/cloudflare to Cloudflare. The public registry lists a few thousand of them, so almost anything with an API is within reach. terraform init is the command that fetches the ones your configuration asks for.

$ terraform init

Initializing the backend...
 
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 5.0"...
- Installing hashicorp/aws v5.70.0...
- Installed hashicorp/aws v5.70.0 (signed by HashiCorp)
 
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
 
Terraform has been successfully initialized!
 
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

$ ls -lh .terraform/providers/registry.terraform.io/hashicorp/aws/5.70.0/linux_amd64/

total 601M
-rwxr-xr-x 1 ops ops 601M Jul 20 09:14 terraform-provider-aws_v5.70.0_x5

That 601 MB file is an ordinary Linux executable sitting in a hidden folder under your working directory. During a plan or an apply, Terraform starts it as a child process, the plugin prints one handshake line saying which local address to dial, and the two of them talk over gRPC (a way for two programs to call each other's functions across a socket) on a local socket, wrapped in TLS (Transport Layer Security, the same encryption your browser uses) with a throwaway certificate minted for that single run. None of that traffic leaves the machine. Now the part that matters for you: the provider is the piece holding your cloud credentials. It reads AWS_ACCESS_KEY_ID, or AWS_PROFILE, or the instance metadata service, or whatever else the shell hands it. Terraform has no login of its own and no permission model of its own. Whoever can run terraform apply in that directory can do everything those credentials allow, with nothing standing between them and the API. Write that on a sticky note and put it on the monitor.

# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.
 
provider "registry.terraform.io/hashicorp/aws" {
  version     = "5.70.0"
  constraints = "~> 5.0"
  hashes = [
    "h1:kQ5ZQF9Zr9k1n0mYQ3WQ9YyG3JcVYQhFq8pWv7Vu2sE=",
    "zh:0f2a1d0f4a1cfe0bf0f2a1c2e3d4b5a69788c0d1e2f3a4b5c6d7e8f9a0b1c2d3",
    "zh:1a2b3c4d5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6b7c8d9e0f1a2b",
    # ... one zh: entry per published platform package
  ]
}

That lock file is your supply chain control, the tamper-evident seal on the bottle. version records exactly which release was chosen. hashes records what the contents must fingerprint to. The h1: line is a fingerprint of the unpacked package as it sits on disk. Each zh: line is the SHA-256 (Secure Hash Algorithm, 256-bit, a function that turns any file into a fixed fingerprint written as 64 hex characters) of one release zip, copied from HashiCorp's signed checksum file. On the next terraform init, a package whose fingerprint is not on that list is refused outright. So commit the file. If your laptop is an arm64 Mac and your CI runners (continuous integration, the automated system that builds and tests every commit) are amd64 Linux, record both platforms with terraform providers lock -platform=linux_amd64 -platform=darwin_arm64. Skip that and the first pipeline run on a platform you never locked can stop dead with a checksum error, and the reflex fix people reach for is deleting the lock file, which throws away exactly the protection you set up.

What Cloud-Agnostic Actually Buys You

People hear cloud-agnostic and expect their AWS configuration to run on Azure by flipping a variable. It does not, and no honest tool claims otherwise. What travels is the tool, the language, the plan-then-apply habit, the state model, and your own skills. What stays put is the resource definitions, because aws_security_group and azurerm_network_security_group are different products with different fields and different defaults. The practical win is still large. One review process and one audit trail covering Amazon, Azure, Kubernetes, Cloudflare, Okta and your GitHub organisation, instead of five consoles with five separate change histories and no way to compare any of them.

Plan First, and Keep the Plan

$ terraform plan -out=tfplan.binary

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create
 
Terraform will perform the following actions:
 
  # aws_security_group.bastion will be created
  + resource "aws_security_group" "bastion" {
      + arn                    = (known after apply)
      + description            = "SSH to the bastion from the office only"
      + egress                 = [
          + {
              + cidr_blocks      = [
                  + "0.0.0.0/0",
                ]
              + description      = ""
              + from_port        = 0
              + ipv6_cidr_blocks = []
              + prefix_list_ids  = []
              + protocol         = "-1"
              + security_groups  = []
              + self             = false
              + to_port          = 0
            },
        ]
      + id                     = (known after apply)
      + ingress                = [
          + {
              + cidr_blocks      = [
                  + "203.0.113.10/32",
                ]
              + description      = ""
              + from_port        = 22
              + ipv6_cidr_blocks = []
              + prefix_list_ids  = []
              + protocol         = "tcp"
              + security_groups  = []
              + self             = false
              + to_port          = 22
            },
        ]
      + name                   = "bastion-ssh"
      + name_prefix            = (known after apply)
      + owner_id               = (known after apply)
      + revoke_rules_on_delete = false
      + tags_all               = (known after apply)
      + vpc_id                 = "vpc-0a1b2c3d4e5f60718"
    }
 
Plan: 1 to add, 0 to change, 0 to destroy.
 
───────────────────────────────────────────────────────────────────────────── 
 
Saved the plan to: tfplan.binary
 
To perform exactly these actions, run the following command to apply:
    terraform apply "tfplan.binary"

A saved plan is a builder's written quote. It says what will be done, and you can hold the builder to it. Four symbols carry most of the meaning. + creates. ~ updates in place. - destroys. And -/+ destroys and then recreates, which on a database or a stateful volume means data loss and downtime, and is the one that should stop a reviewer dead. (You will also see +/-, the same thing with the replacement built before the old one goes, which is gentler but still a replacement.) Read the last line first: Plan: 1 to add, 0 to change, 0 to destroy. Saving the plan with -out is a change-control move, because terraform apply tfplan.binary carries out exactly the actions somebody reviewed, not a fresh diff worked out after a colleague pushed while you were reading. Handle that file with care. It holds variable values and resource attributes in the clear, secrets included, so it has no business being published as a pipeline artifact.

Drift Is Free Detection If You Switch It On

Your housemate drank the milk. Translated to production: at 02:40 an on-call engineer opened SSH to the whole internet from the web console to unblock an incident, and mentioned it to nobody. Or somebody holding stolen console credentials did the same thing and hoped nobody would look. Either way, reality no longer matches your code. Terraform calls that drift, and here is the part most teams never turn on. Terraform will report it on a schedule, for free, without touching a single resource. A refresh-only plan reads the live API, compares it against the recorded state, and tells you what moved without offering to fix anything.

$ terraform plan -refresh-only -detailed-exitcode -input=false -no-color -lock=false; echo "rc=$?"

aws_security_group.bastion: Refreshing state... [id=sg-0f1e2d3c4b5a69788]
 
Note: Objects have changed outside of Terraform
 
Terraform detected the following changes made outside of Terraform since the
last "terraform apply" which may have affected this plan:
 
  # aws_security_group.bastion has been changed
  ~ resource "aws_security_group" "bastion" {
        id      = "sg-0f1e2d3c4b5a69788"
      ~ ingress = [
          + {
              + cidr_blocks      = [
                  + "0.0.0.0/0",
                ]
              + description      = ""
              + from_port        = 22
              + ipv6_cidr_blocks = []
              + prefix_list_ids  = []
              + protocol         = "tcp"
              + security_groups  = []
              + self             = false
              + to_port          = 22
            },
            # (1 unchanged element hidden)
        ]
        name    = "bastion-ssh"
        # (9 unchanged attributes hidden)
    }
 
 
This is a refresh-only plan, so Terraform will not take any actions to undo
these. If you were expecting these changes then you can apply this plan to
record the updated values in the Terraform state without changing any remote
objects.
rc=2

-detailed-exitcode turns the answer into something a machine can act on. 0 means state and reality agree. 1 means the run itself broke. 2 means there are differences. -lock=false stops this read-only check from taking the state lock and blocking a real deployment that happens to start at the same moment. Now notice the difference between the two kinds of plan, because teams get this wrong and then wonder why the alert is useless. A plain terraform plan answers 'what would I change to make reality match the code', which blends your unmerged edits together with other people's out-of-band changes and buries the signal in noise. terraform plan -refresh-only answers one question only: did anything move underneath us. For detection you want the second one.

#!/usr/bin/env bash
# No 'set -e' here on purpose: we want the exit code, not an abort.
set -uo pipefail
 
out=$(terraform -chdir=/srv/infra/prod plan -refresh-only -detailed-exitcode \
      -input=false -no-color -lock=false 2>&1)
rc=$?
 
case "$rc" in
  0) logger -t tf-drift -p user.info    "prod: no drift" ;;
  2) logger -t tf-drift -p user.warning "prod: DRIFT DETECTED"
     printf '%s\n' "$out" | logger -t tf-drift -p user.warning ;;
  *) logger -t tf-drift -p user.err     "prod: drift check failed (rc=$rc)"
     printf '%s\n' "$out" | logger -t tf-drift -p user.err
     exit 1 ;;
esac

[Unit]
Description=Terraform drift check (prod)
Wants=network-online.target
After=network-online.target
 
[Service]
Type=oneshot
User=tfdrift
Group=tfdrift
WorkingDirectory=/srv/infra/prod
# Credentials come from the host's instance role, which is read-only.
Environment=TF_IN_AUTOMATION=1
Environment=AWS_REGION=eu-west-1
ExecStart=/usr/local/bin/tf-drift-check.sh
 
# It only ever needs to read the cloud and write inside its own working dir.
NoNewPrivileges=yes
PrivateTmp=yes
ProtectSystem=strict
ProtectHome=read-only
ReadWritePaths=/srv/infra/prod

[Unit]
Description=Nightly Terraform drift check
 
[Timer]
OnCalendar=*-*-* 03:15:00
RandomizedDelaySec=300
Persistent=true
 
[Install]
WantedBy=timers.target

$ sudo systemctl daemon-reload
$ sudo systemctl enable --now tf-drift.timer
$ systemctl list-timers tf-drift.timer

Created symlink /etc/systemd/system/timers.target.wants/tf-drift.timer → /etc/systemd/system/tf-drift.timer.
NEXT                        LEFT          LAST PASSED UNIT           ACTIVATES
Wed 2026-07-22 03:15:00 UTC 9h 12min left n/a  n/a    tf-drift.timer tf-drift.service
 
1 timers listed.
Pass --all to see loaded but inactive timers, too.

The unit runs as a dedicated account holding a read-only cloud role, so the worst it can do on its worst day is look. ProtectSystem=strict and PrivateTmp keep it out of the rest of the filesystem, ReadWritePaths gives it back the one directory it genuinely needs, and NoNewPrivileges stops anything it launches from gaining more than it started with. Its output lands in journald (the systemd logging service) tagged tf-drift, where your log shipper is already watching, and a user.warning line becomes an alert like any other. Drift stops being something you find out about three weeks later during an audit. Terraform has now told you that something changed and what it was. Your cloud provider's own audit log tells you the rest: who, from where, and with which credentials. On AWS that log is CloudTrail, and a console-made change looks like this, trimmed to the fields that matter.

{
  "eventVersion": "1.09",
  "eventTime": "2026-07-21T02:41:07Z",
  "eventSource": "ec2.amazonaws.com",
  "eventName": "AuthorizeSecurityGroupIngress",
  "awsRegion": "eu-west-1",
  "sourceIPAddress": "198.51.100.24",
  "userAgent": "AWS Internal",
  "sessionCredentialFromConsole": "true",
  "readOnly": false,
  "userIdentity": {
    "type": "AssumedRole",
    "principalId": "AROAEXAMPLEID123456:alice",
    "arn": "arn:aws:sts::123456789012:assumed-role/AdminAccess/alice",
    "accountId": "123456789012",
    "sessionContext": {
      "attributes": {
        "creationDate": "2026-07-21T02:38:44Z",
        "mfaAuthenticated": "true"
      }
    }
  },
  "requestParameters": {
    "groupId": "sg-0f1e2d3c4b5a69788",
    "ipPermissions": {
      "items": [
        {
          "ipProtocol": "tcp",
          "fromPort": 22,
          "toPort": 22,
          "ipRanges": { "items": [ { "cidrIp": "0.0.0.0/0" } ] }
        }
      ]
    }
  }
}

Two fields do the work. userAgent is 'AWS Internal' and sessionCredentialFromConsole is 'true', and together they say a human clicked this in a browser. Compare that against the same change made by Terraform, where the user agent is long and unmistakable: APN/1.0 HashiCorp/1.0 Terraform/1.9.8 (+https://www.terraform.io) terraform-provider-aws/5.70.0 aws-sdk-go-v2/1.30.3 os/linux lang/go/1.22.7. Once every legitimate change to production comes out of your pipeline, console-shaped writes become a small, alertable set, and you can build a detection rule on that shape alone. So keep the discipline that makes the rule work. One hand-made change and your code stops being the truth. The next apply may quietly revert an emergency fix, or refuse to run, or turn up as a surprise -/+ replace on a database nobody expected to lose. Read in the console as much as you like. Change through the code. If an incident genuinely demands a manual fix, treat it as a documented exception and, before the shift ends, either put it into the code or take it back out. A team half committed to IaC collects both sets of problems and neither set of benefits.

Terraform Builds the Kitchen, Something Else Stocks It

Terraform provisions. It creates the machine, the network, the disk, the load balancer, the DNS name, the IAM role (Identity and Access Management, the cloud's own permission system, which decides who may call which API). It is not built to manage what happens inside a running machine: installing packages, editing /etc/ssh/sshd_config, restarting a service. That job is configuration management, and its tools are Ansible, Chef, Puppet, Salt, or cloud-init on first boot. Terraform builds the kitchen. Ansible stocks the shelves and lights the oven. A common shape is Terraform laying down the network and the instances, a Packer-built image carrying the hardened base, and Ansible handling whatever is left. Knowing where that line sits keeps you away from provisioner "remote-exec" blocks, which fire shell commands over SSH from whichever machine happened to run Terraform, record nothing in state so Terraform can never tell you whether what they did is still true, and are documented by HashiCorp itself as a last resort. They are how people get burned.

Do this on Monday, in a repository you already own. Run terraform plan -refresh-only -detailed-exitcode -lock=false against production and read the exit code. If it comes back 0, wire it into a timer while it is still cheap to set up. If it comes back 2, read every line before you touch anything, because what is on your screen is a list of every change somebody made to production without telling the code.

Try this

Run ./provision.sh # clean account: the group is created, the rule comes back as JSON on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: terraform init downloads other people's code and then runs it. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related