The module toolkit

terraform, http-helper, aws, k8s.

A builder hands you the keys and a folder of paperwork. The folder says the wiring is certified, the pipes hold pressure, the front door locks. A good home inspector ignores the folder. They plug a tester into a socket, run the tap, and try the handle from outside in the rain. Terratest gives you both halves of that. The terraform module reads the builder's paperwork. Every other module in the library is a piece of the inspector's kit.

You have already used the terraform module to apply a configuration and read a value back out. This lesson covers the rest of the kit you will reach for constantly: http-helper for knocking on the door, aws for interrogating the cloud provider itself, and k8s for reaching inside a Kubernetes cluster. Knowing what already ships matters more than it sounds. Every helper you have not heard of turns into glue you write by hand, usually by shelling out to a command line tool and scraping its text with a regular expression that breaks the week somebody changes the output format.

Four Modules, Four Different Questions

These four do not stack on top of each other. They are separate instruments, and each answers a question the others cannot. The terraform module answers "what did I ask for". http-helper answers "is anything actually serving traffic". aws answers "what does the cloud provider say is true right now". k8s answers "what is running inside the cluster". Nearly every weak infrastructure test comes from pointing one instrument at a question it was never built to answer, then trusting the reading. A test that checks a Terraform output is non-empty and stops there has measured Terraform's opinion of itself.

You do not have to memorise any of this. Terratest ships as ordinary Go source code, so the Go toolchain is your index. go list -m prints the version you are pinned to, which matters enormously here, because Terratest is midway through renaming most of its surface. go doc prints the exact signature and doc comment for any function, with no browser involved. Get in the habit of running it before you write the call rather than after the call fails.

# which version am I actually testing against?
cd test && go list -m github.com/gruntwork-io/terratest
 
# read the signature before you write the call
go doc github.com/gruntwork-io/terratest/modules/http-helper HttpGetWithRetry

github.com/gruntwork-io/terratest v1.0.1
 
package http_helper // import "github.com/gruntwork-io/terratest/modules/http-helper"
 
func HttpGetWithRetry(t testing.TestingT, url string, tlsConfig *tls.Config,
	expectedStatus int, expectedBody string, retries int,
	sleepBetweenRetries time.Duration)
    HttpGetWithRetry repeatedly performs an HTTP GET on the given URL until the
    given status code and body are returned or until max retries has been
    exceeded.
 
    Deprecated: Use [HTTPGetWithRetryContext] instead.

Two things in that output are worth slowing down for. The first is the package line. The folder on disk is http-helper, but the package inside declares itself http_helper, because a hyphen is not a legal character in a Go identifier. The compiler reads the package clause rather than the folder name, so an unaliased import still compiles. Terratest's own examples spell the alias out anyway (http_helper "github.com/gruntwork-io/terratest/modules/http-helper") so that readers and linters are not tripped up by a package whose name disagrees with the last piece of its path. The same quirk lives in modules/test-structure, whose package is test_structure.

The second is that last line. In the 1.0 releases, almost every classic helper carries a Deprecated: marker pointing at a Context twin. Output became OutputContext, HttpGetE became HTTPGetContextE, GetAccountId became GetAccountID and then GetAccountIDContext. The old spellings still compile and still work, because each one now calls the new one with a background context. They also make go vet and every editor put a strikethrough through your code. New tests should be written in the Context form, and this lesson is, so if you land here from an older tutorial that says terraform.InitAndApply, the mapping is mechanical: add Context to the name and ctx as the second argument.

terraform: One Struct Is One Command Line

terraform.Options is an order form for a single Terraform invocation. Fill it in once, hand it to every call. TerraformDir is the working directory. Vars becomes one -var flag per entry, VarFiles becomes -var-file, Targets becomes -target. EnvVars sets environment variables for the child process, which is where credentials and AWS_DEFAULT_REGION normally live. BackendConfig becomes -backend-config on init. NoColor: true adds -no-color so your continuous integration logs (the automated build server that runs your tests on every push) are not full of escape sequences. Lock is a plain boolean that defaults to false, which is why the apply and destroy lines in the log below carry -lock=false. Harmless when every test run owns its own state file, and a nasty surprise when two runs share a backend. There is also TerraformBinary, which you set to "tofu" when the thing you actually run is OpenTofu.

Wrapping that struct in terraform.WithDefaultRetryableErrors costs one line and is worth doing by default. It merges in a map of known transient failures (a provider plugin download that hiccups, a dropped Kubernetes or Helm connection, a provider that reads back an inconsistent result after apply) and sets the retry policy to three retries five seconds apart, which is four attempts in total. It does not cover cloud API throttling, so add RequestLimitExceeded and ThrottlingException patterns of your own. A bad thirty seconds on the network no longer fails your build. Reading values back is OutputContext for a string, OutputListContext and OutputMapContext for the structured kinds, OutputJSONContext when you want the raw JSON text to unmarshal yourself. All of them run terraform output -json underneath, and that reads state. State is Terraform's own record of what it believes it did. If somebody deleted the bucket by hand ten minutes ago, OutputContext will still hand you its name, cheerfully.

package test
 
import (
	"context"
	"fmt"
	"strings"
	"testing"
	"time"
 
	"github.com/gruntwork-io/terratest/modules/aws"
	http_helper "github.com/gruntwork-io/terratest/modules/http-helper"
	"github.com/gruntwork-io/terratest/modules/random"
	"github.com/gruntwork-io/terratest/modules/terraform"
	"github.com/stretchr/testify/require"
)
 
// The only account this suite is ever allowed to touch.
const sandboxAccountID = "111122223333"
 
func TestWebServer(t *testing.T) {
	t.Parallel()
 
	// One deadline for the whole test. Every Context helper below honours it.
	ctx, cancel := context.WithTimeout(context.Background(), 25*time.Minute)
	defer cancel()
 
	// Guard rail: refuse to apply anything outside the sandbox account.
	require.Equal(t, sandboxAccountID, aws.GetAccountIDContext(t, ctx),
		"wrong AWS credentials loaded; refusing to apply")
 
	region := aws.GetRandomStableRegionContext(t, ctx,
		[]string{"eu-west-1", "us-east-2"}, nil)
	name := "tt-" + strings.ToLower(random.UniqueID())
	instanceType := aws.GetRecommendedInstanceTypeContext(t, ctx, region,
		[]string{"t3.micro", "t2.micro"})
 
	opts := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
		TerraformDir: "../examples/web-server",
		Vars: map[string]any{
			"name":          name,
			"instance_type": instanceType,
		},
		EnvVars: map[string]string{"AWS_DEFAULT_REGION": region},
		NoColor: true,
	})
 
	// WithoutCancel: if ctx times out, teardown must still be allowed to run.
	defer terraform.DestroyContext(t, context.WithoutCancel(ctx), opts)
	terraform.InitAndApplyContext(t, ctx, opts)
 
	// 1. Is anything serving traffic at all? (http-helper)
	url := terraform.OutputContext(t, ctx, opts, "url")
	http_helper.HTTPGetWithRetryContext(t, ctx, url, nil,
		200, "Hello, World!", 30, 5*time.Second)
 
	// 2. What does EC2 itself say about the box? (aws)
	instanceID := terraform.OutputContext(t, ctx, opts, "instance_id")
	tags := aws.GetTagsForEc2InstanceContext(t, ctx, region, instanceID) // region FIRST
	require.Equal(t, "true", tags["terratest"])
	require.NotEmpty(t,
		aws.GetPublicIPOfEc2InstanceContext(t, ctx, instanceID, region)) // region SECOND
 
	// 3. Is the asset bucket configured the way we claim it is?
	bucket := terraform.OutputContext(t, ctx, opts, "bucket_id")
	aws.AssertS3BucketVersioningExistsContext(t, ctx, region, bucket)
	require.Contains(t,
		aws.GetS3ObjectContentsContext(t, ctx, region, bucket, "index.html"), "Hello")
 
	// 4. And is it private? Ask with no credentials at all.
	publicURL := fmt.Sprintf("https://%s.s3.%s.amazonaws.com/index.html", bucket, region)
	status, _, err := http_helper.HTTPGetContextE(t, ctx, publicURL, nil)
	require.NoError(t, err)
	require.Equal(t, 403, status, "bucket %s answers anonymous GETs", bucket)
}

Three lines earn their keep before a single resource exists. aws.GetAccountIDContext asks the Security Token Service (the small AWS service whose entire job is answering "whose credentials are these") which account you are in. Comparing that against a hardcoded sandbox number is the cheapest brake there is on a test that would otherwise run a real apply in production because somebody had the wrong profile exported. random.UniqueID() returns a short random string that goes into every resource name, so two runs never collide and any orphan left behind is traceable to a run. aws.GetRecommendedInstanceTypeContext asks EC2 (Elastic Compute Cloud, the service that rents you virtual machines) which of your candidate sizes actually exists in whichever region you landed in, because t2.micro is not offered everywhere and a hardcoded instance type is a slow, expensive way to discover that.

# real deploy, real money, real teardown
cd test && go test -v -timeout 30m -run TestWebServer

=== RUN   TestWebServer
=== PAUSE TestWebServer
=== CONT  TestWebServer
TestWebServer 2026-07-22T09:14:07Z region.go:128: Using region eu-west-1
TestWebServer 2026-07-22T09:14:09Z retry.go:159: terraform [init -upgrade=false -no-color]
TestWebServer 2026-07-22T09:14:09Z logger.go:79: Running command terraform with args [init -upgrade=false -no-color]
TestWebServer 2026-07-22T09:14:16Z logger.go:79: Terraform has been successfully initialized!
TestWebServer 2026-07-22T09:14:16Z retry.go:159: terraform [apply -input=false -auto-approve -var instance_type=t3.micro -var name=tt-8fk2qz -no-color -lock=false]
TestWebServer 2026-07-22T09:14:16Z logger.go:79: Running command terraform with args [apply -input=false -auto-approve -var instance_type=t3.micro -var name=tt-8fk2qz -no-color -lock=false]
TestWebServer 2026-07-22T09:17:51Z logger.go:79: Apply complete! Resources: 11 added, 0 changed, 0 destroyed.
TestWebServer 2026-07-22T09:17:51Z logger.go:79: Running command terraform with args [output -no-color -json url]
TestWebServer 2026-07-22T09:17:52Z retry.go:159: HTTP GET to URL http://tt-8fk2qz-alb-1043277.eu-west-1.elb.amazonaws.com
TestWebServer 2026-07-22T09:17:52Z http_helper.go:101: Making an HTTP GET call to URL http://tt-8fk2qz-alb-1043277.eu-west-1.elb.amazonaws.com
TestWebServer 2026-07-22T09:17:52Z retry.go:173: HTTP GET to URL http://tt-8fk2qz-alb-1043277.eu-west-1.elb.amazonaws.com returned an error: Get "http://tt-8fk2qz-alb-1043277.eu-west-1.elb.amazonaws.com": dial tcp: lookup tt-8fk2qz-alb-1043277.eu-west-1.elb.amazonaws.com: no such host. Sleeping for 5s and will try again.
... 9 more attempts: first no such host, then Response status: 503 while the target group turns healthy ...
TestWebServer 2026-07-22T09:18:44Z retry.go:159: HTTP GET to URL http://tt-8fk2qz-alb-1043277.eu-west-1.elb.amazonaws.com
TestWebServer 2026-07-22T09:18:44Z http_helper.go:101: Making an HTTP GET call to URL http://tt-8fk2qz-alb-1043277.eu-west-1.elb.amazonaws.com
TestWebServer 2026-07-22T09:18:45Z logger.go:79: Running command terraform with args [output -no-color -json instance_id]
TestWebServer 2026-07-22T09:18:47Z logger.go:79: Running command terraform with args [output -no-color -json bucket_id]
TestWebServer 2026-07-22T09:18:52Z http_helper.go:101: Making an HTTP GET call to URL https://tt-8fk2qz-assets.s3.eu-west-1.amazonaws.com/index.html
TestWebServer 2026-07-22T09:18:53Z logger.go:79: Running command terraform with args [destroy -auto-approve -input=false -var instance_type=t3.micro -var name=tt-8fk2qz -no-color -lock=false]
TestWebServer 2026-07-22T09:21:36Z logger.go:79: Destroy complete! Resources: 11 destroyed.
--- PASS: TestWebServer (449.31s)
PASS
ok  	github.com/example/infra/test	449.362s

Seven and a half minutes, and it was not free. Terratest fakes nothing. It built a load balancer, an instance, a bucket and eight other real resources in a real account at real hourly rates, then deleted them. Two habits keep the bill sane. Put the defer teardown line directly above the apply, so it is registered before anything can fail; a failed assertion unwinds the goroutine and deferred calls still run, which is why the destroy above appears even in the failing runs later in this lesson. Then learn where that guarantee stops. defer does not run when the process dies. Press Ctrl-C, or let -timeout 30m expire, and Go dumps every goroutine stack and exits without unwinding a thing. Your load balancer is still there in the morning. That is also why the deferred destroy above gets context.WithoutCancel: hand it an expired context and teardown refuses to start, leaking the very resources it exists to remove.

http-helper: Knocking Until Somebody Answers

A bare http.Get against a load balancer that was created ninety seconds ago is like ringing a doorbell once, hearing nothing, and driving home. The apply finished, but the Domain Name System record that turns the load balancer's hostname into an address has not reached your resolver, and no backend has passed a health check yet. HTTPGetWithRetryContext keeps knocking. You give it the address, a TLS config (Transport Layer Security, the encryption behind the s in HTTPS), the status code you expect, the body you expect, how many retries to make, and how long to sleep between them. Thirty and five seconds is the common setting. Read that count carefully, because the number is retries, not attempts: the loop runs from zero through thirty inclusive, so you get 31 calls and a little over two and a half minutes of patience.

The body check is stricter than most people assume. The helper trims whitespace off each end of the response and then compares the whole thing for equality. It is not a substring search. The moment a colleague wraps your greeting in an HTML tag, or a template starts appending a footer, the check fails while the service is perfectly healthy. It fails slowly, too, because the retry loop runs all the way to exhaustion first. When you want "contains", or one field out of a JSON document, or anything conditional, reach for HTTPGetWithRetryWithCustomValidationContext and hand it a function that takes the status code and the body and returns a bool.

# same test, after someone wrapped the greeting in markup
cd test && go test -v -timeout 30m -run TestWebServer

TestWebServer 2026-07-22T10:02:44Z retry.go:159: HTTP GET to URL http://tt-p4m7ra-alb-9915042.eu-west-1.elb.amazonaws.com
TestWebServer 2026-07-22T10:02:44Z http_helper.go:101: Making an HTTP GET call to URL http://tt-p4m7ra-alb-9915042.eu-west-1.elb.amazonaws.com
TestWebServer 2026-07-22T10:02:44Z retry.go:173: HTTP GET to URL http://tt-p4m7ra-alb-9915042.eu-west-1.elb.amazonaws.com returned an error: Validation failed for URL http://tt-p4m7ra-alb-9915042.eu-west-1.elb.amazonaws.com. Response status: 200. Response body:
<html><body><h1>Hello, World!</h1></body></html>. Sleeping for 5s and will try again.
... the identical three lines, 30 more times, five seconds apart ...
    http_helper.go:284: 'HTTP GET to URL http://tt-p4m7ra-alb-9915042.eu-west-1.elb.amazonaws.com' unsuccessful after 30 retries
TestWebServer 2026-07-22T10:05:20Z logger.go:79: Running command terraform with args [destroy -auto-approve -input=false -var instance_type=t3.micro -var name=tt-p4m7ra -no-color -lock=false]
TestWebServer 2026-07-22T10:08:01Z logger.go:79: Destroy complete! Resources: 11 destroyed.
--- FAIL: TestWebServer (392.77s)
FAIL
exit status 1
FAIL	github.com/example/infra/test	392.822s

The one-line failure at the bottom is what fools people. 'HTTP GET to URL ...' unsuccessful after 30 retries is the entire failure message, and it says nothing about the cause. The retry loop returns a MaxRetriesExceeded error that has thrown the underlying reason away, and it does that by design, because after thirty-one different failures there is no single reason left to report. The real explanation sits in the Sleeping for 5s lines above, in thirty-one identical copies. So when a Terratest run dies inside a retry loop, scroll up instead of reading the last line and guessing. Scroll by message text, not by the file:line prefix in front of it. That prefix is where inside Terratest the line was emitted rather than where you are in your own test, which is why one number like logger.go:79 fronts output from Terraform, kubectl and the Kubernetes client alike. Those numbers shift when you bump the version, so never match on them. Here the status was 200 on the very first attempt and the body never changed. Retrying harder was never going to help. The fix is a validation function, not a bigger number.

// Instead of an exact-body match, validate the response however you like.
http_helper.HTTPGetWithRetryWithCustomValidationContext(t, ctx, url, nil,
	30, 5*time.Second,
	func(status int, body string) bool {
		return status == 200 && strings.Contains(body, "Hello, World!")
	})
 
// One request, no retry. The status comes back as a value, and err is
// non-nil only when the request itself failed (DNS, TCP, TLS).
status, body, err := http_helper.HTTPGetContextE(t, ctx, url, nil)

aws: Ask the Provider, Not the State File

The aws module talks to the AWS software development kit directly and hands back typed Go values: an address as a string, tags as a map[string]string, the bytes of an object. No subprocess, no JSON parsing, no jq. This is your independent second opinion on everything Terraform told you. Most functions take the region as an explicit argument rather than quietly reading it from the environment, and that is deliberate, because a test that checks the right resource in the wrong region is worse than one that fails outright. GetRandomStableRegionContext picks one from a list you approve, which spreads parallel runs around instead of piling every test into us-east-1 until you hit an account limit.

The security assertions that matter most tend to be the negative ones. Encryption is on. Versioning is on. The bucket is not readable by the public. The first two have purpose-built helpers, such as AssertS3BucketVersioningExistsContext. The third needs no helper at all, because the honest way to prove a bucket is private is to walk up to the front door as a stranger. Ask for the object with no credentials and require a refusal. HTTPGetContextE never signs its requests, so it really is an anonymous visitor even though your test process is holding valid AWS credentials at the time. It gives you the status code as a value and returns an error only when the request itself failed, so a 403 arrives as something you can assert on rather than something that kills the test. When no helper exists for what you need, aws.NewS3ClientContext(t, ctx, region) hands you the underlying SDK client and every call it supports.

# a week later: someone "fixed" a broken CloudFront origin by loosening the bucket
cd test && go test -v -timeout 30m -run TestWebServer

TestWebServer 2026-07-22T11:41:02Z logger.go:79: Apply complete! Resources: 11 added, 0 changed, 0 destroyed.
TestWebServer 2026-07-22T11:41:05Z http_helper.go:101: Making an HTTP GET call to URL http://tt-k93bqx-alb-2280913.eu-west-1.elb.amazonaws.com
TestWebServer 2026-07-22T11:41:08Z http_helper.go:101: Making an HTTP GET call to URL https://tt-k93bqx-assets.s3.eu-west-1.amazonaws.com/index.html
    web_server_test.go:71: 
        	Error Trace:	/home/dev/infra/test/web_server_test.go:71
        	Error:      	Not equal: 
        	            	expected: 403
        	            	actual  : 200
        	Test:       	TestWebServer
        	Messages:   	bucket tt-k93bqx-assets answers anonymous GETs
TestWebServer 2026-07-22T11:41:09Z logger.go:79: Running command terraform with args [destroy -auto-approve -input=false -var instance_type=t3.micro -var name=tt-k93bqx -no-color -lock=false]
TestWebServer 2026-07-22T11:43:52Z logger.go:79: Destroy complete! Resources: 11 destroyed.
--- FAIL: TestWebServer (301.44s)
FAIL
exit status 1
FAIL	github.com/example/infra/test	301.489s

That is exactly the failure you want, and notice how little else noticed. The plan was clean. The apply succeeded with eleven resources added. Every Terraform output held the same value as the day before, so an assertion on outputs would have sailed straight through. Nothing in state records the difference between a private bucket and a world-readable one, because from Terraform's point of view both are simply the configuration it was handed. The only thing that caught it was a request made from outside the account, carrying no credentials, that got a 200 where it should have got a 403.

k8s: Reaching Inside the Cluster

k8s.NewKubectlOptions(contextName, configPath, namespace) is a card with your kubectl flags written on it, and you hand the same card to every function in the module. Empty strings mean "use the default", so NewKubectlOptions("", "", "payments") means the current context in the default kubeconfig file, scoped to the payments namespace. Convenient on a laptop, and the same class of mistake as the wrong AWS profile, because "the current context" is whatever you last pointed kubectl config use-context at, and one of those is production. Name the cluster you mean. The other pattern worth copying is a throwaway namespace per run. It stops parallel tests from fighting over object names, and it gives cleanup exactly one object to delete instead of a list of manifests to unwind in the right order.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 2
  selector:
    matchLabels: { app: web }
  template:
    metadata:
      labels: { app: web }
    spec:
      automountServiceAccountToken: false   # the pod needs no API access
      containers:
        - name: web
          image: nginxinc/nginx-unprivileged:1.27-alpine
          ports:
            - containerPort: 8080
          securityContext:
            runAsNonRoot: true
            allowPrivilegeEscalation: false
            capabilities:
              drop: ["ALL"]
---
apiVersion: v1
kind: Service
metadata:
  name: web-svc
spec:
  type: ClusterIP
  selector: { app: web }
  ports:
    - port: 8080
      targetPort: 8080

KubectlApplyContext shells out to kubectl apply -f on that file. WaitUntilServiceAvailableContext polls until the Service object can be fetched, and only for type: LoadBalancer does it go further and wait for the cloud provider to assign an address. GetServiceContext returns the real Kubernetes API object. A ClusterIP Service has no address your test host can dial, so you reach it through a port-forward. k8s.NewTunnel opens one from a local port on the test machine, through the API server, down to the pods behind the Service, and tunnel.Endpoint() hands you a host:port string. Notice what that string does not contain: a scheme. Feed it straight to http-helper and Go's HTTP client rejects it with unsupported protocol scheme "", which reads like a Terratest bug and is not one. Prefix it with http:// yourself. Note also that namespaces are created and deleted through the Kubernetes API client rather than the kubectl binary, so those two steps produce no Running command kubectl line in the log at all.

RunKubectlAndGetOutputContextE is the escape hatch for everything the module does not wrap, and it is how you write access-control tests. RBAC (role-based access control) is the rulebook that decides which identity is allowed to perform which action in a cluster. kubectl auth can-i get secrets --as system:serviceaccount:ns:default asks the API server directly whether one identity may take one action. That is a far stronger check than reading back the Role you applied, because it runs the same evaluation code that will one day answer an attacker holding that service account's token. The manifest above already sets automountServiceAccountToken: false, so no token is mounted into the pod in the first place. Two locks on the same door. The manifest fits the first one, and the test proves the second.

// same imports as before, plus "github.com/gruntwork-io/terratest/modules/k8s"
 
// Name the cluster you mean. Never inherit whatever context happens to be current.
const kubeContext = "arn:aws:eks:eu-west-1:111122223333:cluster/tt-ci"
 
func TestClusterWorkload(t *testing.T) {
	ctx, cancel := context.WithTimeout(context.Background(), 25*time.Minute)
	defer cancel()
 
	namespace := "tt-" + strings.ToLower(random.UniqueID())
	options := k8s.NewKubectlOptions(kubeContext, "", namespace)
 
	k8s.CreateNamespaceContext(t, ctx, options, namespace)
	defer k8s.DeleteNamespaceContext(t, context.WithoutCancel(ctx), options, namespace)
 
	k8s.KubectlApplyContext(t, ctx, options, "../manifests/web.yml")
	k8s.WaitUntilServiceAvailableContext(t, ctx, options, "web-svc", 20, 5*time.Second)
 
	// ClusterIP, so nothing is reachable from the test host: tunnel in instead.
	// Local port 0 lets the operating system pick a free one, which parallel runs need.
	tunnel := k8s.NewTunnel(options, k8s.ResourceTypeService, "web-svc", 0, 8080)
	defer tunnel.Close() // registered last, so it runs first
	tunnel.ForwardPort(t)
 
	http_helper.HTTPGetWithRetryWithCustomValidationContext(t, ctx,
		"http://"+tunnel.Endpoint(), nil, 30, 5*time.Second,
		func(status int, body string) bool {
			return status == 200 && strings.Contains(body, "Welcome to nginx")
		})
 
	// The workload's identity must NOT be able to read Secrets.
	// kubectl exits 1 when the answer is "no", so this needs the E variant.
	out, err := k8s.RunKubectlAndGetOutputContextE(t, ctx, options,
		"auth", "can-i", "get", "secrets",
		"--as", "system:serviceaccount:"+namespace+":default")
	require.Error(t, err, "kubectl exited 0, so the answer was yes")
	require.Contains(t, out, "no")
}

# needs a live cluster and a working kubeconfig context
cd test && go test -v -timeout 30m -run TestClusterWorkload

=== RUN   TestClusterWorkload
TestClusterWorkload 2026-07-22T12:03:08Z logger.go:79: Configuring Kubernetes client using config file /home/dev/.kube/config with context arn:aws:eks:eu-west-1:111122223333:cluster/tt-ci
TestClusterWorkload 2026-07-22T12:03:09Z logger.go:79: Running command kubectl with args [--context arn:aws:eks:eu-west-1:111122223333:cluster/tt-ci --namespace tt-4qz8mv apply -f ../manifests/web.yml]
TestClusterWorkload 2026-07-22T12:03:10Z logger.go:79: deployment.apps/web created
TestClusterWorkload 2026-07-22T12:03:10Z logger.go:79: service/web-svc created
TestClusterWorkload 2026-07-22T12:03:10Z retry.go:159: Wait for service web-svc to be provisioned.
TestClusterWorkload 2026-07-22T12:03:10Z logger.go:79: Service is now available
TestClusterWorkload 2026-07-22T12:03:10Z logger.go:79: Creating a port forwarding tunnel for resource service/web-svc routing local port 0 to remote port 8080
TestClusterWorkload 2026-07-22T12:03:11Z logger.go:79: Selected port 42817
TestClusterWorkload 2026-07-22T12:03:11Z retry.go:159: HTTP GET to URL http://localhost:42817
TestClusterWorkload 2026-07-22T12:03:11Z http_helper.go:101: Making an HTTP GET call to URL http://localhost:42817
TestClusterWorkload 2026-07-22T12:03:12Z retry.go:173: HTTP GET to URL http://localhost:42817 returned an error: Get "http://localhost:42817": dial tcp 127.0.0.1:42817: connect: connection refused. Sleeping for 5s and will try again.
... two more attempts, while the two pods finish starting ...
TestClusterWorkload 2026-07-22T12:03:27Z http_helper.go:101: Making an HTTP GET call to URL http://localhost:42817
TestClusterWorkload 2026-07-22T12:03:28Z logger.go:79: Running command kubectl with args [--context arn:aws:eks:eu-west-1:111122223333:cluster/tt-ci --namespace tt-4qz8mv auth can-i get secrets --as system:serviceaccount:tt-4qz8mv:default]
TestClusterWorkload 2026-07-22T12:03:29Z logger.go:79: no
TestClusterWorkload 2026-07-22T12:03:30Z logger.go:79: Configuring Kubernetes client using config file /home/dev/.kube/config with context arn:aws:eks:eu-west-1:111122223333:cluster/tt-ci
--- PASS: TestClusterWorkload (24.18s)
PASS
ok  	github.com/example/infra/test	24.233s

Every Helper Has a Twin That Hands You the Error

Almost every function in Terratest exists twice: Foo(t, ctx, ...) and FooE(t, ctx, ...). The plain one is a thin wrapper that calls the E one and fails the test on any error it gets back, usually through require.NoError, which stops the test dead at the first sign of trouble. You never see the error value, and the line after the call never runs. The E one returns (result, error) and lets you decide what the failure means. That single letter causes more confusion than anything else in the library, because a plain call does not look like it can end your test, and it can, from a line inside Terratest that you did not write and will meet for the first time in a stack trace.

The access-control check is the cleanest example. kubectl auth can-i exits with status 1 when the answer is no, and no is the answer you are hoping for. There is a further wrinkle in the k8s module: the plain, non-E form is RunKubectlContext, and it returns nothing at all, so it gives you neither the error nor the output. Swap it in for the E variant and the failing exit becomes a dead test. The same reasoning covers terraform.InitAndApplyContextE when you expect a policy to block the apply, aws.GetPublicIPOfEc2InstanceContextE when an address may not be assigned yet, and HTTPGetContextE when a 403 is the pass condition.

# the same check written with RunKubectlContext (no trailing E)
cd test && go test -v -timeout 30m -run TestClusterWorkload

TestClusterWorkload 2026-07-22T12:31:04Z logger.go:79: Running command kubectl with args [--context arn:aws:eks:eu-west-1:111122223333:cluster/tt-ci --namespace tt-9wd3kp auth can-i get secrets --as system:serviceaccount:tt-9wd3kp:default]
TestClusterWorkload 2026-07-22T12:31:05Z logger.go:79: no
    kubectl.go:16: 
        	Error Trace:	/home/dev/go/pkg/mod/github.com/gruntwork-io/[email protected]/modules/k8s/kubectl.go:16
        	            	/home/dev/infra/test/cluster_test.go:31
        	Error:      	Received unexpected error:
        	            	error while running command: exit status 1; 
        	Test:       	TestClusterWorkload
--- FAIL: TestClusterWorkload (98.21s)
FAIL
exit status 1
FAIL	github.com/example/infra/test	98.267s

Read that slowly. kubectl printed no, which is the correct and desirable answer, and the test failed anyway. An engineer debugging this at five o'clock on a Friday will "fix" it by deleting the assertion, and the suite quietly stops checking access control forever. So here is the rule of thumb. Use the plain form on the happy path, where any failure should stop the test loudly and immediately. Use the E form the moment a failure is the expected result, or you want to run the retry loop yourself, or you want to attach a message that explains what a green test actually proved. Security assertions live almost entirely in that second category, which is why a suite written only from plain calls nearly always turns out to be a suite that checks things exist and nothing else.

Try this

Run go doc github.com/gruntwork-io/terratest/modules/http-helper HttpGetWithRetry on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: turning off certificate checks turns off the test. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related