>_SecOpsLog_
CoursesTerratestAssertions & retries

Assertions & retries

Eventual consistency in tests.

Advanced14 min · lesson 5 of 12

Your test applies a module that builds a load balancer, reads the DNS name out of a Terraform output, and sends one web request. It fails. You run the same test again forty seconds later, change nothing, and it passes. Nothing got fixed. Your test showed up early.

Cloud APIs (application programming interfaces, the machine-facing front doors of a cloud service) behave like a company directory. HR presses save and the new hire officially exists, right then. Reception's phone list has not caught up. Neither has the badge reader at the loading dock, nor the printout taped inside the supply closet. Ask any of those and you are told nobody by that name works here. Ask again in a minute and the answer changes, with no new action from anyone. That lag between written and visible-everywhere is eventual consistency (the system will agree with itself, but not immediately).

terraform apply returns when the control plane (the API layer that accepts your changes and files them) has accepted the change, not when the world has finished acting on it. An ALB (application load balancer, the AWS component that spreads incoming traffic across several servers) exists the moment the API says created. Its DNS (domain name system, the internet's phone book) name may not resolve for another minute. It answers 503 until at least one target passes a health check, and with Terraform's aws_lb_target_group defaults of a thirty second check interval and three consecutive successes, that is a minute to ninety seconds after your container starts answering. A brand new IAM (identity and access management, the AWS permissions system) role often cannot be assumed for a few seconds, and the API reports that as AccessDenied, which reads exactly like a broken policy. A pod that Kubernetes has scheduled still has to pull an image across the network. None of that is a bug. It is the ordinary shape of a distributed system, and your assertion is a visitor who knocked before the door was hung.

Poll, Do Not Sleep

The tempting fix is time.Sleep(60 * time.Second). It is a guess, and it manages to be wrong in both directions at once. On a good run the load balancer was ready in eight seconds and you burned fifty two doing nothing, on every run, on every branch, with billable infrastructure idling the whole time. Forty tests carrying one sixty second sleep each is forty minutes of pipeline spent staring at a wall. On a bad run the load balancer needed sixty three seconds, the test failed anyway, somebody bumps the sleep to ninety, and the suite gets slower without getting more reliable.

Polling is what you already do with a kettle. You do not stand there for a fixed four minutes. You glance at it and pour when it clicks. retry.DoWithRetryContext is that glance, in a loop: run your function, and if it returns an error, wait a fixed interval and run it again, until it succeeds or the attempts run out. Success returns straight away, so the good path stays fast. Failure costs the full budget and then reports a clear timeout instead of a mystery.

One naming note before the code, because it will trip you up in any suite written more than a few months ago. Terratest v1.0 deprecated every function that did not accept a context.Context, and fixed the initialism casing on the way through. terraform.InitAndApply became terraform.InitAndApplyContext, retry.DoWithRetry became retry.DoWithRetryContext, and the HTTP helpers moved from HttpGet... to HTTPGet...Context. The old names still compile, because they now forward to the new ones with context.Background(), but every linter you own will flag them. Everything below uses the current names.

test/alb_test.go
package test
 
import (
	"context"
	"fmt"
	"net/http"
	"testing"
	"time"
 
	"github.com/gruntwork-io/terratest/modules/retry"
	"github.com/gruntwork-io/terratest/modules/terraform"
)
 
func TestAlbServesTraffic(t *testing.T) {
	// Plain background context. Go 1.24 added t.Context(), which is cancelled
	// just before t.Cleanup functions run: fine for polling and for a defer in
	// this function, wrong for teardown registered with t.Cleanup, which would
	// get an already-cancelled context.
	ctx := context.Background()
 
	opts := &terraform.Options{TerraformDir: "../examples/alb"}
 
	defer terraform.DestroyContext(t, ctx, opts)
	terraform.InitAndApplyContext(t, ctx, opts)
 
	url := "http://" + terraform.OutputContext(t, ctx, opts, "alb_dns_name")
 
	// Go's default HTTP client has NO timeout. One connection that opens and
	// then goes quiet can outlast the entire retry budget below.
	client := &http.Client{Timeout: 10 * time.Second}
 
	retry.DoWithRetryContext(t, ctx, "GET "+url, 30, 5*time.Second, func() (string, error) {
		resp, err := client.Get(url)
		if err != nil {
			// No DNS record yet, connection refused, handshake failed: all worth another look.
			return "", err
		}
		defer resp.Body.Close()
 
		if resp.StatusCode == 404 {
			// Waiting cannot invent a path that does not exist. Stop the loop now.
			return "", retry.FatalError{Underlying: fmt.Errorf("404 from %s", url)}
		}
		if resp.StatusCode != 200 {
			return "", fmt.Errorf("status %d, want 200", resp.StatusCode)
		}
		return "ok", nil
	})
}

After t and the context, the arguments are a description for the logs, the maximum number of retries, the wait between attempts, and the function to run. Two details bite people. First, maxRetries counts retries, not attempts: the loop is for i := 0; i <= maxRetries; i++, so 30 means up to 31 calls. Second, Terratest sleeps after every failed attempt, including the last one, so the worst case is roughly 31 times the sum of your function's own duration and the sleep. With a five second gap and a ten second client timeout, that single check can hold the test for nearly eight minutes before it gives up. Write that number down. You will need it at the end of this lesson.

terminal
# 30m budget: a real apply plus a real destroy do not fit in Go's 10m default
go test -v -timeout 30m -run TestAlbServesTraffic ./test/
output
=== RUN   TestAlbServesTraffic
TestAlbServesTraffic 2026-07-22T09:14:02Z command.go:200: Running command terraform with args [apply -input=false -auto-approve -lock=false]
TestAlbServesTraffic 2026-07-22T09:16:38Z command.go:301: Apply complete! Resources: 23 added, 0 changed, 0 destroyed.
TestAlbServesTraffic 2026-07-22T09:16:38Z command.go:200: Running command terraform with args [output -no-color -json alb_dns_name]
TestAlbServesTraffic 2026-07-22T09:16:39Z retry.go:159: GET http://api-tt-7fk2wq-1893204551.eu-west-1.elb.amazonaws.com
TestAlbServesTraffic 2026-07-22T09:16:39Z retry.go:173: GET http://api-tt-7fk2wq-1893204551.eu-west-1.elb.amazonaws.com returned an error: Get "http://api-tt-7fk2wq-1893204551.eu-west-1.elb.amazonaws.com": dial tcp: lookup api-tt-7fk2wq-1893204551.eu-west-1.elb.amazonaws.com on 10.0.0.2:53: no such host. Sleeping for 5s and will try again.
TestAlbServesTraffic 2026-07-22T09:16:45Z retry.go:159: GET http://api-tt-7fk2wq-1893204551.eu-west-1.elb.amazonaws.com
TestAlbServesTraffic 2026-07-22T09:16:45Z retry.go:173: GET http://api-tt-7fk2wq-1893204551.eu-west-1.elb.amazonaws.com returned an error: status 503, want 200. Sleeping for 5s and will try again.
[... 8 more attempts, every one a 503: no healthy target behind the load balancer yet ...]
TestAlbServesTraffic 2026-07-22T09:17:34Z retry.go:159: GET http://api-tt-7fk2wq-1893204551.eu-west-1.elb.amazonaws.com
TestAlbServesTraffic 2026-07-22T09:17:35Z command.go:200: Running command terraform with args [destroy -auto-approve -input=false -lock=false]
TestAlbServesTraffic 2026-07-22T09:19:46Z command.go:301: Destroy complete! Resources: 23 destroyed.
--- PASS: TestAlbServesTraffic (344.29s)
PASS
ok      github.com/acme/infra/test      344.851s

Read the timestamps, because they are the measurement. The first attempt fails because the name does not resolve yet. The next nine collect a 503 from a load balancer with nothing healthy behind it. The eleventh, at 09:17:34, gets a 200 and the loop returns. The wait cost fifty five seconds and it cost exactly fifty five seconds, not a padded ninety. A sleep tuned for this stack would have to spend at least that long on every run forever, including the runs where it was ready in five.

The Helper That Already Wraps the Loop

You rarely need to hand-roll that closure for HTTP (hypertext transfer protocol, the request-and-response language browsers and servers speak). The http-helper module ships the same loop with the request already written, and its log lines are the ones you will see most often in a Terratest suite.

test/alb_test.go
import (
	"context"
 
	http_helper "github.com/gruntwork-io/terratest/modules/http-helper"
)
 
// The import needs an explicit alias: the directory is http-helper,
// but the Go package inside it is http_helper.
 
// After t and ctx: url, tlsConfig, expectedStatus, expectedBody, retries, sleepBetweenRetries
http_helper.HTTPGetWithRetryContext(t, ctx, url, nil, 200, "Hello, World!", 30, 5*time.Second)

Pass nil for the TLS (transport layer security, the encryption behind https) config to get the defaults. Inside, the helper builds its own client with a ten second timeout, for the reason its own source comment gives: Go does not impose one, so a connection attempt can hang for a very long time. Every attempt logs twice, once as Making an HTTP GET call to URL <your url> from the helper itself and once as HTTP GET to URL <your url> from the retry loop wrapped around it. That pair is how you spot these attempts in a wall of test output.

The trap is expectedBody. It is not a substring search. Terratest trims the whitespace off both ends of the response and then compares the whole thing with ==, so a stray newline is forgiven and nothing else is. The day somebody adds a build tag to the page, every attempt fails, and the failure looks exactly like an infrastructure problem.

terminal
go test -v -timeout 30m -run TestAlbServesTraffic ./test/
output
=== RUN   TestAlbServesTraffic
TestAlbServesTraffic 2026-07-22T09:22:17Z command.go:200: Running command terraform with args [apply -input=false -auto-approve -lock=false]
TestAlbServesTraffic 2026-07-22T09:24:57Z command.go:301: Apply complete! Resources: 23 added, 0 changed, 0 destroyed.
TestAlbServesTraffic 2026-07-22T09:24:57Z command.go:200: Running command terraform with args [output -no-color -json alb_dns_name]
TestAlbServesTraffic 2026-07-22T09:24:58Z retry.go:159: HTTP GET to URL http://api-tt-9dm4rb-1044728310.eu-west-1.elb.amazonaws.com
TestAlbServesTraffic 2026-07-22T09:24:58Z http_helper.go:101: Making an HTTP GET call to URL http://api-tt-9dm4rb-1044728310.eu-west-1.elb.amazonaws.com
TestAlbServesTraffic 2026-07-22T09:24:58Z retry.go:173: HTTP GET to URL http://api-tt-9dm4rb-1044728310.eu-west-1.elb.amazonaws.com returned an error: Validation failed for URL http://api-tt-9dm4rb-1044728310.eu-west-1.elb.amazonaws.com. Response status: 200. Response body:
Hello, World! (build 2f9c1ab). Sleeping for 5s and will try again.
[... 30 more attempts, every single one identical ...]
TestAlbServesTraffic 2026-07-22T09:27:33Z command.go:200: Running command terraform with args [destroy -auto-approve -input=false -lock=false]
TestAlbServesTraffic 2026-07-22T09:29:44Z command.go:301: Destroy complete! Resources: 23 destroyed.
    http_helper.go:284: 'HTTP GET to URL http://api-tt-9dm4rb-1044728310.eu-west-1.elb.amazonaws.com' unsuccessful after 30 retries
--- FAIL: TestAlbServesTraffic (447.12s)
FAIL
FAIL    github.com/acme/infra/test      447.688s

Status 200 on the first attempt. Status 200 on the thirty first. Two and a half minutes of sleeping between them, and the error prints the body it received on its own line, so the answer is sitting right there in the log. Confirm it against the live endpoint before you touch the Terraform.

terminal
curl -s http://api-tt-9dm4rb-1044728310.eu-west-1.elb.amazonaws.com
output
Hello, World! (build 2f9c1ab)

The infrastructure was healthy the entire time. Somebody added a build tag to the greeting and the test now compares two strings that will never be equal. Raising the retry count makes it slower and no truer. Say what you actually mean instead, with the custom validation form.

test/alb_test.go
// Exact match: dies on every attempt once the page gains a build tag.
// http_helper.HTTPGetWithRetryContext(t, ctx, url, nil, 200, "Hello, World!", 30, 5*time.Second)
 
// What you meant: right status, and the greeting is in there somewhere.
http_helper.HTTPGetWithRetryWithCustomValidationContext(t, ctx, url, nil, 30, 5*time.Second,
	func(status int, body string) bool {
		return status == 200 && strings.Contains(body, "Hello, World!")
	})
Retrying cannot turn a permanent failure into a temporary one
When the first attempt and the last attempt report the same status and the same body, waiting was never the answer, and every extra retry is pure cost. That applies to a 403 from a policy, a 404 from a wrong path, and a certificate that does not match the hostname. Return retry.FatalError{Underlying: err} for those so the loop stops in seconds and logs Returning due to fatal error. Watch out for the shortcut people reach for on the certificate case: passing &tls.Config{InsecureSkipVerify: true} to the http-helper functions makes the red test go green by switching off certificate checking entirely, chain and hostname both, which deletes the one assertion that was doing real security work. Fix the certificate or fix the hostname you are testing. Do not fix the thermometer.

Retry the Errors You Meant to Retry

The same idea applies one level up, to Terraform itself. AWS throttles API calls when a burst of parallel tests hits it. A provider plugin occasionally drops its connection mid-download. A provider sometimes reads back stale data straight after a write. terraform.Options carries three fields for exactly this. RetryableTerraformErrors maps a pattern to a human note, MaxRetries sets how many extra attempts to allow, and TimeBetweenRetries sets the gap. The keys are regular expressions, and each one is tested against both the command output and the error text.

test/vpc_test.go
opts := &terraform.Options{
	TerraformDir: "../examples/vpc",
 
	// Keys are REGEXES, matched against the command output and the error text.
	// Values are the note printed when one of them fires.
	RetryableTerraformErrors: map[string]string{
		".*Provider produced inconsistent result after apply.*": "Provider eventual consistency error.",
		".*RequestLimitExceeded.*":                              "AWS API throttling under parallel tests.",
		".*Throttling.*":                                        "AWS API throttling under parallel tests.",
	},
 
	// Without these two the map above does nothing at all: MaxRetries is 0.
	MaxRetries:         3,
	TimeBetweenRetries: 15 * time.Second,
}
 
// Terratest ships a curated list: plugin download failures, dropped Kubernetes
// and Helm connections, and provider eventual-consistency bugs. It does NOT
// include API throttling, so keep your own keys as well.
// It returns a CLONE, and it OVERWRITES MaxRetries with 3 and
// TimeBetweenRetries with 5s, so call it first and set your values after.
opts = terraform.WithDefaultRetryableErrors(t, opts)
opts.MaxRetries = 3
opts.TimeBetweenRetries = 15 * time.Second
terminal
go test -v -timeout 30m -run TestVpcModule ./test/
output
TestVpcModule 2026-07-22T09:31:07Z command.go:200: Running command terraform with args [apply -input=false -auto-approve -lock=false]
TestVpcModule 2026-07-22T09:32:19Z command.go:301: Error: Provider produced inconsistent result after apply
TestVpcModule 2026-07-22T09:32:19Z command.go:301: When applying changes to aws_nat_gateway.this[0], provider "registry.terraform.io/hashicorp/aws" produced an unexpected new value: Root object was present, but now absent.
TestVpcModule 2026-07-22T09:32:19Z retry.go:247: 'terraform [apply -input=false -auto-approve -lock=false]' failed with the error 'error while running command: exit status 1; Error: Provider produced inconsistent result after apply' but this error was expected and warrants a retry. Further details: Provider eventual consistency error.
TestVpcModule 2026-07-22T09:32:19Z retry.go:173: terraform [apply -input=false -auto-approve -lock=false] returned an error: error while running command: exit status 1; Error: Provider produced inconsistent result after apply. Sleeping for 15s and will try again.
TestVpcModule 2026-07-22T09:32:34Z command.go:200: Running command terraform with args [apply -input=false -auto-approve -lock=false]
TestVpcModule 2026-07-22T09:33:58Z command.go:301: Apply complete! Resources: 14 added, 0 changed, 0 destroyed.

MaxRetries is an int, so it starts at zero, and the error map on its own changes nothing. That catches people: the list looks configured, the test still dies on the first throttle. When a pattern matches, Terratest prints which note fired and tries again. When nothing matches, it wraps the error in retry.FatalError and stops on the spot. That is the right default. An error you did not name is an error you did not expect.

Resist the urge to add a catch-all like Error: to that map. A pattern that wide makes AccessDenied retryable, so a genuinely broken IAM policy gets three extra chances to fail slowly instead of one chance to fail clearly, and the test meant to prove your permissions are correct now takes minutes to tell you nothing. There is a real cost on the other side too, and it is worth being honest about it. A retried apply runs again over a partly built stack, and for any resource whose create call succeeded while its response was lost in the network, you can finish with an orphan that neither the Terraform state nor your teardown knows about. Retry apply for throttling and network flakiness. Do not retry it for errors you have not read.

Never Retry a Must-Not Check

Here is where retry quietly turns a security test into theatre. Somebody writes a check that the admin port is closed, wraps a TCP (transmission control protocol, the connection layer underneath HTTP) dial in retry.DoWithRetryContext, and treats connection refused as success. It passes. It also passes on the run where a change opened that port to 0.0.0.0/0 (the whole internet), because the loop keeps looking until it catches a moment when nothing is listening, and while a service is still starting there is always such a moment. A loop that runs until a check succeeds will find what it is looking for.

Positive and negative checks need opposite treatment. A positive check (this should answer) is safe to retry, because the answer only gets more true as the system settles. A negative check (this should not answer, this port should not be open, this bucket should not be public) can only be trusted once the system has finished settling. So prove convergence with a positive control first, then make the negative assertion exactly once, with nothing wrapped around it.

test/alb_test.go
base := "https://" + terraform.OutputContext(t, ctx, opts, "alb_dns_name")
 
// 1. Positive control. Retry here: the answer only becomes more true over time.
http_helper.HTTPGetWithRetryWithCustomValidationContext(t, ctx, base+"/health", nil, 30, 5*time.Second,
	func(status int, body string) bool { return status == 200 })
 
// 2. The stack is up, so a "must not" assertion finally means something.
//    One request. No loop. If it fails, it fails.
status, _ := http_helper.HTTPGetContext(t, ctx, base+"/admin", nil)
require.Equalf(t, 403, status, "/admin answered %d with no session cookie", status)

The rule that keeps this straight is retry the fetch, assert on the result. Put the call that talks to the network or the cloud API inside the retry function, and put the comparison outside it, where it runs once. DoWithRetryContext hands back whatever string your function returned, so the assertion has everything it needs after the loop ends. This is also why a testify require inside the closure misleads: require calls t.FailNow, which ends the test on the spot rather than triggering another attempt, and an assert in there marks the test failed and then keeps spinning through the remaining budget for nothing.

test/alb_test.go
// Fetch inside the loop. It is allowed to be late.
body := retry.DoWithRetryContext(t, ctx, "read /whoami", 10, 3*time.Second, func() (string, error) {
	code, body, err := http_helper.HTTPGetContextE(t, ctx, base+"/whoami", nil)
	if err != nil {
		return "", err
	}
	if code != 200 {
		return "", fmt.Errorf("status %d, want 200", code)
	}
	return body, nil
})
 
// Judge outside the loop. It gets exactly one opinion.
require.NotContains(t, body, "uid=0(root)", "the app must not run as root")
An assertion just failed. Should the test ask again?
Why did this assertion fail?
still converging
Retry, with a budget
DNS not resolving, 503 from an empty target group, a provider reading back stale data
same answer every time
Return retry.FatalError
404, 403, a body that will never match: stop in seconds instead of minutes
a must-not check
Do not loop at all
Gate on a positive control, then assert exactly once
The first two branches are about patience. The third is about honesty: a loop that runs until a check succeeds will always eventually find a moment when it succeeds.

Budgets That Outlive Your Timeout

Every retry budget you write spends wall clock, and go test keeps an alarm clock of its own. Its default is ten minutes, which is nothing once a real apply is involved, and -timeout is not advice. It is a hard kill.

terminal
# no -timeout flag, so go test falls back to its 10 minute default
go test -v -run TestAlbServesTraffic ./test/
output
=== RUN   TestAlbServesTraffic
TestAlbServesTraffic 2026-07-22T10:02:11Z command.go:200: Running command terraform with args [apply -input=false -auto-approve -lock=false]
TestAlbServesTraffic 2026-07-22T10:04:47Z command.go:301: Apply complete! Resources: 23 added, 0 changed, 0 destroyed.
TestAlbServesTraffic 2026-07-22T10:04:48Z retry.go:159: HTTP GET to URL https://api-tt-3xq8vp-0871245529.eu-west-1.elb.amazonaws.com/health
TestAlbServesTraffic 2026-07-22T10:04:48Z http_helper.go:101: Making an HTTP GET call to URL https://api-tt-3xq8vp-0871245529.eu-west-1.elb.amazonaws.com/health
[... the target group never goes healthy; the loop keeps polling ...]
panic: test timed out after 10m0s
	running tests:
		TestAlbServesTraffic (10m0s)
 
goroutine 41 [running]:
testing.(*M).startAlarm.func1()
	/usr/local/go/src/testing/testing.go:2802 +0x385
created by time.goFunc
	/usr/local/go/src/time/sleep.go:215 +0x2d
 
goroutine 1 [chan receive, 10 minutes]:
testing.(*T).Run(0xc0001021c0, {0x8e1f3a?, 0x0?}, 0x904e60)
	/usr/local/go/src/testing/testing.go:1859 +0x431
[... full goroutine dump ...]
exit status 2
FAIL    github.com/acme/infra/test      600.024s

Notice what is missing. No --- FAIL line, and no Destroy complete. Compare that with the failing run earlier in this lesson, where the deferred destroy ran to completion before the failure was even printed. That difference is not luck. A testify require or a Terratest helper giving up calls t.Fatal, which calls runtime.Goexit, and Goexit runs every deferred call on its way out. The timeout panic fires from a separate watchdog goroutine and takes the whole process down with it, so the defers belonging to your test goroutine never run. One kind of failure cleans up after itself. The other walks away.

terminal
aws elbv2 describe-load-balancers --region eu-west-1 \
  --query "LoadBalancers[?starts_with(LoadBalancerName, 'api-tt-')].[LoadBalancerName,CreatedTime,State.Code]" \
  --output text
output
api-tt-3xq8vp   2026-07-22T10:02:44.010000+00:00        active

That load balancer is still up, along with its target group, its security group, and whatever NAT (network address translation) gateway the VPC (virtual private cloud, your own private slice of the provider's network) brought with it. A NAT gateway on its own runs around a dollar a day before it moves a single byte, so the money is the smaller problem. What you have left behind is infrastructure nobody owns: created by a test account with permissions wide enough to build a network, absent from every inventory, outside every patching cycle, and quite possibly reachable from the internet through a security group somebody wrote for a test rather than for production. Orphaned test infrastructure is a boring and entirely real source of incidents, and it is the specific way this lesson's topic hurts you.

Size the budget so Go's alarm never gets to fire
Add up every retry budget in a test, plus the apply and the destroy, and keep the total comfortably under -timeout. Two traps sit either side of that. Setting -timeout 0 disables the alarm completely, which converts a hung test into a CI (continuous integration, the service that runs your tests on every push) job that runs until somebody notices, and CI runners get killed on their own schedule with the same missing cleanup. Meanwhile t.Parallel() puts every parallel test in one process sharing one alarm, so the single slowest test can kill the cleanup for all of them at once. Set your CI job timeout longer than the go test timeout so Go fires first and your deferred destroy still gets its chance, and run a scheduled sweeper that deletes anything carrying your test tag and older than a few hours. One day the runner dies anyway.

Size budgets from measurements rather than from feel, because the logs already hold the answer. The passing run above shows the load balancer converging on the eleventh attempt in fifty five seconds, so a budget of thirty retries leaves comfortable headroom for a slow day and still fails in under three minutes when something is genuinely broken. Re-measure whenever the stack changes shape, and when a check starts eating its whole budget, read the first logged attempt before you raise the number.

Quick check
01Why is a retry loop better than a fixed time.Sleep before an assertion?
Incorrect — Wrong mechanism: time.Sleep only parks its own goroutine, and the retry loop is equally synchronous.
Incorrect — Asking repeatedly does not speed anything up; the load balancer converges at its own pace either way.
Correct — it is fast in the common case and gives you a clear timeout error in the bad case.
Incorrect — Consistency is a property of the service, not of the client; no amount of polling changes it.
02In http_helper.HTTPGetWithRetryContext(t, ctx, url, nil, 200, "Hello, World!", 30, 5*time.Second), how is expectedBody compared with the response?
Correct — the validation function is statusCode == expectedStatusCode && body == expectedBody, where the body was already run through strings.TrimSpace.
Incorrect — That is what people assume; use HTTPGetWithRetryWithCustomValidationContext with strings.Contains if that is what you want.
Incorrect — Regexes are only used for RetryableTerraformErrors keys, never for the HTTP body.
Incorrect — Both conditions are evaluated together in one boolean expression; an empty string is a real expectation of an empty body.
03A teammate's test asserts that the admin port is closed. It wraps a TCP dial in retry.DoWithRetryContext and returns success when the dial is refused, retrying otherwise. It passed on the run where a change opened that port to 0.0.0.0/0. What is the real fix?
Incorrect — A bigger budget makes it worse: more attempts means more chances to catch a moment when nothing is listening.
Correct — retrying a must-not condition guarantees a pass, because it keeps looking until the service happens to be unreachable.
Incorrect — The listener protocol has nothing to do with it, and the test would still be hunting for a passing moment.
Incorrect — That changes when the test stops, not the fact that the loop is searching for a moment the check succeeds.

Try this

Run go test -v -timeout 30m -run TestAlbServesTraffic ./test/ on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: retrying cannot turn a permanent failure into a temporary one. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Cleanup & defer destroyTesting Kubernetes & Helm

Related

What InSpec is: compliance as codeInSpecThe secrets-in-Git problem & SOPSSOPS