Terraform: providers, resources & HCL

The core building blocks.

A parts list and an assembly manual are different documents. One says what the finished thing contains. The other says what to do, and in what order. Terraform files are a parts list. You write down the pieces of infrastructure that should exist, and Terraform works out the ordering and the calls to the cloud on its own. The language you write that list in is HCL (HashiCorp Configuration Language), a small text format built for describing things rather than for programming them.

Three ideas carry most of Terraform. A provider is a plugin that knows how to talk to one platform's API (Application Programming Interface, the machine-to-machine control surface behind a service, the same thing the web console is clicking on your behalf). A resource is one thing you want to exist: a virtual machine, a storage bucket, a DNS record (Domain Name System, the internet's address book), a firewall rule. And .tf files are the plain text files where you declare both. Learn those three properly and you can read almost any Terraform repository, including the one nobody on your team has opened in two years.

terraform {
  required_version = ">= 1.5"       # the Terraform CLI itself, not a provider
 
  required_providers {
    aws = {                         # "aws" is a local nickname for this config
      source  = "hashicorp/aws"     # namespace / type in the registry
      version = "~> 5.0"            # any 5.x, never 6.0
    }
  }
}
 
provider "aws" {
  region = "eu-west-1"              # provider settings. no credentials here
}
 
resource "aws_instance" "web" {     # resource TYPE, then a name you choose
  ami           = "ami-0d3e2c8b6a4f1e9c7"   # AMI ids are per region
  instance_type = "t3.micro"
 
  tags = {                          # an object value, not a nested block
    Name  = "web-01"
    Owner = "platform-team"
  }
}

How An HCL File Is Put Together

HCL has one shape and repeats it until the file runs out. A block starts with a type word (terraform, provider, resource, data, variable, output, module, locals), then zero or more quoted labels, then a body in curly braces. Inside the body you set arguments with name = value, one per line. So resource "aws_instance" "web" is a block of type resource carrying two labels: the resource type, which the provider defines, and a name you pick. Nothing else in the language works differently. Once you see the pattern, unfamiliar Terraform stops looking unfamiliar.

Two details trip people up early. Curly braces mean two different things: a nested block, like ingress { ... } inside a security group, and an object value, like tags = { Name = "web-01" }. The giveaway is the equals sign. An object gets assigned to an argument; a block never does. The second detail is that arguments inside a block are separated by newlines, not commas, so squashing a block onto one line with commas between the arguments will not parse. Comments use #, // or /* */. And terraform fmt rewrites spacing into one canonical style, which keeps diffs small enough that a reviewer reads the change instead of the whitespace.

Providers Are Plugins You Download And Run

A provider is the contractor you hire because they already speak the supplier's language and hold the account. Terraform core knows nothing about EC2 instances (Elastic Compute Cloud, Amazon's rented virtual machines), S3 buckets (Simple Storage Service, its object storage) or DNS zones. It knows how to read HCL, build a graph, and hand instructions to a plugin. That plugin is a separate program sitting on your disk. Terraform launches it as a child process and talks to it over a local socket using gRPC (a remote procedure call protocol, where one program calls a function inside another program and waits for the answer). Each provider ships a catalogue of resource types and data sources, and that catalogue is the entire vocabulary you get.

The source address tells you where the plugin comes from. hashicorp/aws is shorthand for registry.terraform.io/hashicorp/aws, which reads as host, namespace, type. The namespace is the trust boundary. hashicorp/aws is published by HashiCorp; acme-corp/aws would be somebody else's build exposing exactly the same resource names, and a config pointing at it looks nearly identical in a pull request. The aws = { ... } key inside required_providers is only a local nickname, and every aws_ resource in the file is routed to whatever source address that nickname points at. Read source lines with the suspicion you give an image name in a Dockerfile.

$ terraform init

Initializing the backend...
 
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 5.0"...
- Installing hashicorp/aws v5.82.2...
- Installed hashicorp/aws v5.82.2 (signed by HashiCorp)
 
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
 
Terraform has been successfully initialized!

$ cd .terraform/providers/registry.terraform.io/hashicorp/aws/5.82.2/linux_amd64
$ ls -lh
$ file terraform-provider-aws_v5.82.2_x5

total 668M
-rwxr-xr-x 1 dev dev 668M Feb 11 09:22 terraform-provider-aws_v5.82.2_x5
 
terraform-provider-aws_v5.82.2_x5: ELF 64-bit LSB executable, x86-64,
version 1 (SYSV), statically linked, Go BuildID=Qk3ZtA1p..., stripped

That is two thirds of a gigabyte of somebody else's compiled code, and you are about to run it on your laptop and on your build agents with credentials that can create and delete production. Terraform's defence is verification. The registry publishes a checksum file covering every package in a release, official providers are signed with HashiCorp's GPG key (GNU Privacy Guard, the standard tool for signing files), and init refuses to install a package whose checksum does not match what it was told to expect. The registry also labels publishers as official, partner or community. Community means somebody signed up.

# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.
 
provider "registry.terraform.io/hashicorp/aws" {
  version     = "5.82.2"          # what init actually chose
  constraints = "~> 5.0"          # what your config asked for
  hashes = [
    "h1:0uHFZ2mUOJHTOfRfPTsE0ymCX0WhwYnhWEVwd0pZi/A=",
    "zh:0f1ec4c8fbfe5b16fe4b0daaa8f8cb2c94b9d1e8b2ba6f6bbf0e0f8d1c2a0e21",
    # one zh: line per published platform package
  ]
}

Those two version lines do different jobs. The constraint is your permission slip; the lock file is the receipt. The constraint syntax is worth memorising: ~> 5.0 means at least 5.0 and below 6.0, ~> 5.82.0 means at least 5.82.0 and below 5.83.0, and >= 5.0 on its own means anything, forever, including the next major release that renames arguments underneath you. The hashes are what stops a swapped package. On every init, Terraform compares the archive it downloaded against them and fails loudly rather than quietly installing something different from last time.

$ terraform providers

 
Providers required by configuration:
.
└── provider[registry.terraform.io/hashicorp/aws] ~> 5.0
 
Providers required by state:
 
    provider[registry.terraform.io/hashicorp/aws]
 

So pin the upper bound and commit the lock file. An unpinned provider means the next terraform init on any machine can pull a new major version that renamed or removed arguments, which at best breaks the config and at worst proposes changes to live infrastructure nobody asked for. Set version = "~> 5.0", commit .terraform.lock.hcl to Git, and treat any change to that file as a reviewable event, because a diff there means the code that talks to your cloud has changed. Some older repository templates still list the lock file in .gitignore. That throws away the one reproducibility guarantee Terraform hands you for free.

Resources: Arguments In, Attributes Out

A resource block behaves like an order form. Arguments are the boxes you fill in: ami (Amazon Machine Image, the disk image the virtual machine boots from), instance_type, tags. Attributes are what comes back stamped on the form once the platform has processed it: id, arn (Amazon Resource Name, the unique identifier AWS gives every object it creates), private_ip, and dozens more per resource type. You can read an attribute anywhere else in the config, which is what wires the files together instead of leaving you with a pile of separate declarations. Attributes only the platform can decide show up in a plan as (known after apply), because an instance id genuinely does not exist until the API call comes back.

data "aws_ami" "ubuntu" {           # read-only lookup, creates nothing
  most_recent = true
  owners      = ["099720109477"]    # Canonical's account id, not "whoever"
 
  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd-gp3/ubuntu-noble-24.04-amd64-server-*"]
  }
}
 
resource "aws_security_group" "web_sg" {
  name        = "web-sg"
  description = "HTTPS in from the internet"
 
  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]     # every address on the internet
  }
 
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"              # -1 means every protocol
    cidr_blocks = ["0.0.0.0/0"]
  }
}
 
resource "aws_instance" "web" {
  ami                    = data.aws_ami.ubuntu.id          # attribute of a data source
  instance_type          = "t3.micro"
  vpc_security_group_ids = [aws_security_group.web_sg.id]  # attribute of a resource
}

A data block looks like a resource and runs in the opposite direction: it reads something that already exists instead of creating it. This one asks AWS for the newest Ubuntu 24.04 image, and the owners argument is doing security work. Drop it, and most_recent = true will cheerfully pick the newest image from any account whose image name happens to match your filter, so your servers boot a stranger's operating system. Pinning owners to Canonical's account id closes that door. Apply the same instinct to every lookup whose result feeds a resource.

The two labels together form the resource address, aws_instance.web, and that address is the identity Terraform uses everywhere: in state, in plan output, in terraform state show. Names must be unique within a type. Renaming the second label is not a cosmetic edit. Terraform sees one address vanish and a new one appear, then plans a destroy and a create, which on a database is a very bad afternoon. When you do need to rename, add a moved block with from set to the old address and to set to the new one, and Terraform updates the record instead of rebuilding the thing.

References Decide What Gets Built First

Nowhere in that file did you say "create the security group first". You did not need to. The moment aws_instance.web reads aws_security_group.web_sg.id, Terraform records an edge between the two, builds a dependency graph out of all such edges, then walks it so every resource is created after the things it points at and destroyed before them on the way back down. Independent resources run in parallel, ten at a time by default. For a dependency the API does not express as an attribute, such as an IAM (Identity and Access Management, the permissions system) policy that must attach before an instance can use its role, depends_on states the edge by hand.

$ terraform validate

╷
│ Error: Reference to undeclared resource
│
│   on main.tf line 33, in resource "aws_instance" "web":
│   33:   vpc_security_group_ids = [aws_security_group.web.id]
│
│ A managed resource "aws_security_group" "web" has not been declared in the
│ root module.
╵

One wrong word caused that: aws_security_group.web instead of web_sg. It looks like a typo check, and there is a security property hiding inside it. A reference that does not resolve stops the run, so a config can never quietly attach a firewall group that does not exist. A reference that resolves to the wrong existing group validates cleanly and applies happily. Validation proves your file is well formed and internally consistent. It has no idea which rules you meant.

Read It The Way A Reviewer Would

When a Terraform change lands in a pull request, four things earn a slow read. Any cidr_blocks entry of 0.0.0.0/0 (CIDR is Classless Inter-Domain Routing, the notation for a range of addresses, and that particular range is the whole internet), which is ordinary on port 443 and alarming on 22 or 3389, the ports for SSH (Secure Shell, remote command line access) and RDP (Remote Desktop Protocol). Any credential typed into the file, including access_key inside a provider block, because provider credentials belong in environment variables or in a role the runner assumes. Image and package sources with no owner or checksum pinned. And version constraints that lost their upper bound. Spotting all four needs no tool. It needs someone who knows what the blocks mean.

$ terraform fmt -check -recursive   # report only, do not rewrite

main.tf
modules/network/vpc.tf

Those two files are not in canonical format, so the command exits non-zero and a CI job (Continuous Integration, the pipeline that runs on every push) wired to it fails the build until somebody runs terraform fmt. Pair it with terraform validate, which parses the config, checks argument names against the provider's schema and resolves every reference without calling a single cloud API. It does need terraform init to have run first, since the schema comes out of the plugin. Both are shape checks. Neither one knows that SSH open to the internet is a bad idea, which is the job of the policy scanners later in this course.

Lock Every Platform Your Pipeline Runs On

Checksums are recorded per platform, and that detail bites teams. Install straight from the public registry and Terraform reads the signed checksum file for the whole release, so the lock file ends up holding a zh: hash for every package HashiCorp published plus an h1: hash for the one it actually unpacked. Install any other way, from a network or filesystem mirror, a private registry that serves no signed checksums, or a plugin directory you pointed at by hand, and Terraform only ever learns about the package in front of it. Now the lock file describes one platform. Your laptop is darwin_arm64 (macOS on an Apple chip), your build agent is linux_amd64, and the agent finds no hash it can match. Terraform does not guess its way past a checksum. It stops.

$ terraform init      # on the Linux build agent

Initializing the backend...
 
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 5.0"...
- Installing hashicorp/aws v5.82.2...
╷
│ Error: Failed to install provider
│
│ Error while installing hashicorp/aws v5.82.2: the local package for
│ registry.terraform.io/hashicorp/aws 5.82.2 doesn't match any of the
│ checksums previously recorded in the dependency lock file (this might be
│ because the available checksums are for packages targeting different
│ platforms); for more information:
│ https://developer.hashicorp.com/terraform/language/files/dependency-lock#checksum-verification
╵

$ terraform providers lock \
    -platform=linux_amd64 \
    -platform=darwin_arm64

- Fetching hashicorp/aws 5.82.2 for linux_amd64...
- Retrieved hashicorp/aws 5.82.2 for linux_amd64 (signed by HashiCorp)
- Fetching hashicorp/aws 5.82.2 for darwin_arm64...
- Retrieved hashicorp/aws 5.82.2 for darwin_arm64 (signed by HashiCorp)
- Obtained hashicorp/aws checksums for linux_amd64; Additional checksums for
  this platform were added to the lock file
- Obtained hashicorp/aws checksums for darwin_arm64; All checksums for this
  platform were already tracked in the lock file
 
Success! Terraform has updated the lock file.
 
Review the changes in .terraform.lock.hcl and then commit to your
version control system to retain the new checksums.

Commit that lock file change alongside the code change that caused it. Both platforms are now pinned to the same version and the same verified bytes, so a laptop and a build agent install byte-identical plugins. Expect small lock file diffs from CI even when everything is working, because Terraform adds the newer h1: line for a platform the first time it verifies a package there. A diff in .terraform.lock.hcl is something a reviewer can see, which beats an invisible difference between one engineer's machine and the pipeline that touches production.

When you inherit a repository you did not write, read the terraform block before anything else. It tells you which Terraform versions are allowed, which providers this code can reach, from whose namespace, and how tightly pinned. Then check that .terraform.lock.hcl is committed and covers every platform your pipeline runs on. Those two files decide which code executes against your accounts, and they take about thirty seconds to read.

Try this

Run terraform init on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: secrets typed into HCL do not stay in HCL. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related