HCL: providers, resources & references

The core building blocks of a config.

A Terraform file reads like a work order you hand to a builder. Near the top you list the tools the job needs, and which brand and version of each you will accept. Then you name the building you are working in, and hand over the key. Then, line by line, you list what should exist when the work is finished. Those three parts have names: the terraform block, the provider block, and resource blocks. You write all of them in HCL (HashiCorp Configuration Language, a plain text format built for describing infrastructure), and once you can read those three shapes, you can read almost any Terraform configuration you find in a repository.

terraform {
  required_version = ">= 1.5.0"        # which Terraform versions may run this
 
  required_providers {
    aws = {                            # local nickname: what "aws_*" types bind to
      source  = "hashicorp/aws"        # WHO publishes it (the real identity)
      version = "~> 5.0"               # allowed range: >= 5.0.0 and < 6.0.0
    }
  }
}
 
provider "aws" {
  region = "us-east-1"                 # configure the plugin. no credentials here
}
 
resource "aws_instance" "web" {        # TYPE "local label"  ->  address aws_instance.web
  ami           = "ami-0c7217cdde317cfec"
  instance_type = "t3.micro"
 
  tags = { Name = "web-server" }
}

Providers Are Plugins You Download and Run

Terraform by itself knows nothing about Amazon. It is a planner with a language attached. The knowledge of how to build a server or a firewall rule lives in a separate program called a provider, and a provider works like a translator you hire for one country: you give the instructions, the translator does the talking to the local officials in their own language. Each one speaks a single platform's API (application programming interface, the control surface a service exposes for programs rather than for people clicking buttons). There is one for AWS (Amazon Web Services), one for Azure, one for Google Cloud, one for Kubernetes, one for GitHub, one for Cloudflare, and several thousand more on the public registry. That is why one tool and one language cover almost anything with an API. The workflow never changes. Only the plugin does.

required_providers is where you name the ones you need, and each entry has two halves that do very different jobs. The label on the left, aws, is a nickname you chose, and its only power is that every resource type starting with aws_ binds to it inside this configuration. source is the real identity: hashicorp/aws is shorthand for registry.terraform.io/hashicorp/aws, meaning the aws provider published under the hashicorp namespace. version is a range rather than a fixed choice. The ~> operator, read aloud as pessimistic, lets the rightmost number you wrote move and freezes everything to its left, so ~> 5.0 means at least 5.0.0 and below 6.0.0.

Leave required_providers out entirely and Terraform still works, which is the trap. It sees aws_instance, assumes hashicorp/aws from the type prefix, and installs the newest version in existence. Your supply chain is then decided by whatever somebody published this morning. Write the block, even for a throwaway config.

terraform init

Initializing the backend...
 
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 5.0"...
- Installing hashicorp/aws v5.68.0...
- Installed hashicorp/aws v5.68.0 (signed by HashiCorp)
 
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
 
Terraform has been successfully initialized!
 
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

Read that as a receipt. Terraform asked the registry which 5.x versions exist, took the newest one the constraint allowed, downloaded it, checked the publisher's signature, and wrote a lock file. The plugin is now sitting in a hidden directory beside your code. Go and look at it.

ls -lh .terraform/providers/registry.terraform.io/hashicorp/aws/5.68.0/linux_amd64/

total 549M
-rw-r--r-- 1 ubuntu ubuntu  17K Nov 12 11:02 LICENSE.txt
-rwxr-xr-x 1 ubuntu ubuntu 549M Nov 12 11:02 terraform-provider-aws_v5.68.0_x5

That is a 549 megabyte executable that a command you typed thirty seconds ago fetched off the internet and dropped on your disk. The x5 on the end of the filename is the plugin protocol version it speaks. Run terraform plan and Terraform starts that binary as a child process, then talks to it over gRPC (a way for one running program to call functions inside another) on a local socket. The plugin is the half that holds your cloud credentials and makes the API calls. So terraform init does not run that code, it stages it, and the very next command you type does run it. On a CI runner (continuous integration, the automation that builds and checks every change), the machine doing all this often carries the most powerful cloud role your team owns. A pull request that edits a source line or a lock file is a change to what executes with those permissions, and it deserves the same attention as a change to your application's dependencies.

The Provider Block Is the Key, Not the Keyring

A provider block configures a provider you have already declared: which region, which API endpoint, sometimes which role to assume. Credentials are the part people get wrong. Every mature provider hunts for them in a fixed order, and for AWS that order runs static values written into the block, then environment variables such as AWS_ACCESS_KEY_ID, then the shared file at ~/.aws/credentials, then the container or instance role the platform hands out on its own. You want the last one. Nothing is stored on disk, the credentials are short lived, and they expire without anybody having to remember to rotate them. Static values in the block are the option you never want, and they are also the easiest to find.

grep -rnE '^\s*(access_key|secret_key)\s*=' --include='*.tf' .

./modules/legacy-vpc/provider.tf:4:  access_key = "AKIAIOSFODNN7EXAMPLE"
./modules/legacy-vpc/provider.tf:5:  secret_key = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

Two hits in a module nobody has touched in a year, and both are now in every clone of the repository and in its whole Git history. Rotating the key is the fix. Deleting the line is not, because the old commit still holds it. An AWS access key ID is exactly twenty characters, starts with AKIA, and uses nothing but capital letters and digits, which makes it one of the easiest patterns in existence to alert on, so a secret scanner in CI catches this in seconds.

While you have the provider blocks open, look for alias as well. An alias lets one configuration hold several copies of the same provider, typically one per account or region, and any resource carrying a line like provider = aws.dr is being built somewhere other than the default. That one line is often the difference between a change landing in staging and landing in production.

Resources Are the Things That Should Exist

A resource block declares one thing you want to be real. Its header carries two quoted strings that do different jobs. In resource "aws_instance" "web", the first is a type fixed by the provider, and the part before its first underscore is what binds it to a provider nickname. The second is a local label you invented, unique only within that type. Together they form the resource address, aws_instance.web, and that address is the name Terraform uses in state (the file where Terraform records what it has already built), in plan output, on the command line, and in every reference. Inside the braces, most lines are arguments the provider defines. Five names are reserved by Terraform itself and mean the same thing on every resource anywhere: count, for_each, provider, depends_on, and lifecycle.

Terraform loads every file ending in .tf in the working directory and merges them into one configuration. Order does not matter, and splitting things into main.tf, variables.tf and outputs.tf is habit rather than rule. A data block looks almost identical to a resource and does the opposite: Terraform reads something that already exists and never touches it. Machine images are the everyday case. An AMI (Amazon Machine Image, the frozen copy of a disk that a new server boots from) has an ID that points at one build from one particular day. Hard-code ami = "ami-0c7217cdde317cfec" and six months later that is an Ubuntu image missing six months of security patches, redeployed quietly every time an instance is replaced. A data block asks the provider for the current image instead, filtered to a publisher you trust by account number rather than by a name anyone can copy.

data "aws_ami" "ubuntu" {
  most_recent = true
  owners      = ["099720109477"]        # Canonical's AWS account ID, not "anything named ubuntu"
 
  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"]
  }
}
 
resource "aws_security_group" "web_sg" {
  name        = "web-sg"
  description = "HTTPS in from the internet"
 
  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]         # public on purpose. on port 22 this is a finding
  }
 
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"                  # -1 means every protocol
    cidr_blocks = ["0.0.0.0/0"]
  }
}
 
resource "aws_instance" "web" {
  ami                    = data.aws_ami.ubuntu.id           # ref -> data block resolves first
  instance_type          = "t3.micro"
  vpc_security_group_ids = [aws_security_group.web_sg.id]    # ref -> group created first
 
  tags = { Name = "web-server" }
}

The address is the identity, so tidying up a name is not a cosmetic edit. Change resource "aws_instance" "web" to resource "aws_instance" "web_server" and Terraform does not see a rename. It sees aws_instance.web missing from the configuration and aws_instance.web_server missing from the world, and it plans one destroy plus one create. The running server is torn down and a new one is built in its place. On a database, that is your data. Tell Terraform they are the same thing by adding a moved block naming the old address as from and the new one as to, or by running terraform state mv, then confirm the plan reports no changes before you apply.

References Decide the Order

You never tell Terraform to build the security group first. You write aws_security_group.web_sg.id where the instance needs a group, and the ordering falls out of that by itself. A reference is an address followed by an attribute: aws_security_group.web_sg.id for a resource, data.aws_ami.ubuntu.id for a data block (note the leading data), var.region for an input variable, local.name_prefix for a computed local, module.network.vpc_id for a module output. Draw an arrow from every reference to the thing it points at and you have a wall chart of the whole build. Terraform builds exactly that chart, a DAG (directed acyclic graph, a map of what depends on what, with no loops permitted), then walks it, starting anything whose dependencies are finished and running ten of them at a time by default.

terraform plan | grep -E 'will be created|vpc_security_group_ids'

  # aws_instance.web will be created
      + vpc_security_group_ids               = (known after apply)
  # aws_security_group.web_sg will be created

That phrase, (known after apply), is the graph made visible. Terraform cannot print the group's ID because the group does not exist yet, and the ID only comes back from the AWS API at the moment it is created. Every attribute showing those words is an attribute that something else has to produce first. So when you read a plan, the unknown values are the ordering map. The reverse is worth watching for too. An attribute you expected to be unknown showing up as a literal string usually means somebody pasted an ID in where a reference belonged.

terraform apply

  Enter a value: yes
 
aws_security_group.web_sg: Creating...
aws_security_group.web_sg: Creation complete after 4s [id=sg-0a1b2c3d4e5f67890]
aws_instance.web: Creating...
aws_instance.web: Still creating... [10s elapsed]
aws_instance.web: Still creating... [20s elapsed]
aws_instance.web: Still creating... [30s elapsed]
aws_instance.web: Creation complete after 33s [id=i-0abc123def4567890]
 
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Group first, instance second, and nothing in the file asked for that. Some dependencies are real but invisible to Terraform, though. An instance that uses an IAM (identity and access management, the permissions system) instance profile can be created the instant the profile exists, while the policy that makes the profile useful is still attaching, so the machine boots without the access it needs and the failure looks random. No attribute reference describes that relationship, so you state it outright with depends_on = [aws_iam_role_policy_attachment.web_ssm]. Use it only where a reference genuinely cannot express the link, because every unnecessary depends_on makes the graph more serial and the apply slower. And if two resources end up referencing each other, the graph has a loop, and Terraform stops with a Cycle error naming both addresses. That is a design problem in your configuration, not a bug in the tool.

A mistyped reference is the most common way all of this breaks, and catching it costs nothing. terraform validate parses the whole configuration and checks every reference against every declared block. It makes no API calls and needs no credentials. It does need the directory initialized first, because it checks your arguments against the provider's schema, which means it belongs on every pull request right after init.

terraform validate

╷
│ Error: Reference to undeclared resource
│
│   on main.tf line 39, in resource "aws_instance" "web":
│   39:   vpc_security_group_ids = [aws_security_group.web.id]
│
│ A managed resource "aws_security_group" "web" has not been declared in the
│ root module.
╵

The Constraint Is a Range, the Lock File Is the Pin

Pinning is two separate mechanisms, and treating them as one is where teams get surprised. version = "~> 5.0" is a range, so a fresh init next month can happily pick up 5.69 or 5.80, with new defaults, new validation, and arguments that behave differently. .terraform.lock.hcl is the actual pin. Terraform writes it the first time it resolves providers, recording the exact version it settled on plus a list of cryptographic checksums for the published packages. Every later init in that directory, on any machine, reads the lock file first and installs precisely that build without asking the registry what is new. That is why the file belongs in Git, and why a clean clone on a build server produces the same plan as your laptop.

# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.
 
provider "registry.terraform.io/hashicorp/aws" {
  version     = "5.68.0"      # the exact build every run will use
  constraints = "~> 5.0"      # the range that was allowed when this was resolved
  hashes = [
    "h1:je8ULdKMhb4kkYTOEwLBNXPYYPxDPjKmVLDcqbYxTMs=",
    "zh:3330c0d49fb329dff6de17913e1a774e75aa0913106c3197814c73c3a12a4c3f",
    "zh:9b12af85486a96aedd8d7984b0ff811a4b42e3d88dad1a3fb4c0b580d04fa425",
    # one zh: entry per published platform package
  ]
}

The hashes are an integrity check, not decoration. Treat them as tamper tape on a parcel. The h1: entry is a hash of the extracted package as it sits on disk. The zh: entries are hashes of the official zip files for every platform, taken from a checksum list that HashiCorp signs with a GPG (GNU Privacy Guard, the standard tool for signing files and verifying signatures) key. On each init, Terraform recomputes and compares. If the bytes it fetched do not match what the lock file remembers, whether because an internal mirror was tampered with or a package was quietly republished, it refuses to continue.

terraform init

Initializing the backend...
 
Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Installing hashicorp/aws v5.68.0...
╷
│ Error: Failed to install provider
│
│ Error while installing hashicorp/aws v5.68.0: the current package for
│ registry.terraform.io/hashicorp/aws 5.68.0 doesn't match any of the
│ checksums previously recorded in the dependency lock file; for more
│ information:
│ https://developer.hashicorp.com/terraform/language/files/dependency-lock#checksum-verification
╵

That failed init is the control working, so resist the urge to make it quiet. There are two honest ways out: fix the mirror or the package, or run terraform init -upgrade, which re-resolves inside the constraint and rewrites the lock file with whatever it finds. Reach for -upgrade on purpose, in its own commit, never as a reflex when init complains. Then read the resulting diff in review the way you would read a change to an application's dependency file. A new version number and a fresh set of hashes means different code will run against your cloud account, holding your credentials, the next time somebody types apply.

Try this

Run terraform init on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: the name on the left is cosmetic. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related