IaC pipelines & config

Reviewed/scanned IaC; Key Vault & App Config.

A Bicep file sitting on your laptop is a blueprint taped to a lamppost. Technically it is a plan. It changes nothing. Cities don't work that way. You hand blueprints to a permitting office, an inspector marks exactly what will change, someone signs the permit, and a licensed crew does the build. An IaC pipeline (infrastructure as code, run by your CI system) is that permitting office for your Azure estate. The last two lessons handed you the blueprint languages. This one is the machinery that reviews them, approves them, applies them, and keeps the alarm codes (your secrets) off the blueprint entirely.

Why apply never happens on a laptop

Four words carry this whole topic. A plan (the Terraform word) or a what-if (the Bicep and Azure Resource Manager word) is a dry run: the engine works out what *would* change without touching anything. An apply executes a plan for real. Drift is the gap that opens when somebody changes infrastructure outside the code, like a portal tweak at 2 a.m. that the repo never hears about. And idempotency means running the same deployment twice leaves you in the same place, which is what makes automatic re-runs safe. Apply from a laptop and you inherit three problems at once. Whose credentials ran it. Which version of the code actually went out. And no record that answers either question. A pipeline squashes all three into one identity, one commit SHA (the unique fingerprint Git gives every commit), and one audit log entry per change.

The shape that falls out of that is boring and correct. Every change under infra/ opens a PR (pull request, the review step before code merges). The PR runs a preview plus an IaC scan (the scanners themselves get their own DevSecOps lesson). A human reads the preview. Production applies wait behind an approval. A scheduled job re-runs the preview to catch drift. Boring is the point. Surprises belong in the preview, never in the apply.

Preview before apply: what-if and terraform plan

az deployment group what-if sends your compiled template to Azure Resource Manager (ARM, the service that actually creates and updates every resource in Azure). ARM compares it against live resource state on its own servers and hands back a typed diff: create, modify, delete, no change. Nothing is deployed. Because ARM computes that diff itself, it knows about defaults and server-set properties a tool on your laptop cannot see. It is not perfect. Some resource providers report noise, properties flagged ~ Modify that will not actually change, so read a what-if the way you read a code review, not the way you read a contract.

# Ask ARM what would change — nothing is applied
az deployment group what-if \
  --resource-group rg-orders-prod \
  --template-file infra/main.bicep \
  --parameters infra/prod.bicepparam
 
# Resource and property changes are indicated with these symbols:
#   + Create
#   ~ Modify
#
# The deployment will update the following scope:
#
# Scope: /subscriptions/2e9f.../resourceGroups/rg-orders-prod
#
#   ~ Microsoft.Web/sites/app-orders-prod [2024-04-01]
#     ~ properties.siteConfig.linuxFxVersion: "DOTNETCORE|8.0" => "DOTNETCORE|9.0"
#
#   + Microsoft.KeyVault/vaults/kv-orders-prod [2023-07-01]
#
# Resource changes: 1 to create, 1 to modify.

Terraform reaches the same place through different plumbing. terraform plan compares your configuration against the state file, a JSON inventory of every resource Terraform manages. On Azure that state belongs in a Storage Account: versioned, locked with a blob lease so two applies cannot collide, and authenticated with Entra ID (Microsoft's cloud identity service, formerly Azure AD) instead of a shared storage key. Write the plan out to a file and apply *that file*, and production gets exactly the change somebody reviewed, not whatever main happens to look like ten minutes later.

# infra/backend.tf — state lives in Azure Storage, not on a laptop
terraform {
  backend "azurerm" {
    resource_group_name  = "rg-tfstate"
    storage_account_name = "sttfstateorders"
    container_name       = "tfstate"
    key                  = "orders-prod.tfstate"
    use_azuread_auth     = true   # Entra ID token, no storage account key
  }
}
 
# --- in the pipeline ---
terraform init
terraform plan -out=tfplan -detailed-exitcode
#   # azurerm_key_vault_secret.sql_conn will be created
#   # azurerm_linux_web_app.orders will be updated in-place
# Plan: 1 to add, 1 to change, 0 to destroy.
# exit code: 2  (0 = no changes, 1 = error, 2 = diff pending — gate on this)
 
terraform apply tfplan   # applies exactly the reviewed plan, nothing newer
# Apply complete! Resources: 1 added, 1 changed, 0 destroyed.

The pipeline: plan on PR, apply on main

Two stages do the work. Preview runs what-if and posts the diff for a human to read. Apply runs only on main, and it uses a deployment job pointed at an environment. That is the load-bearing detail, because environments are where Azure DevOps attaches approvals and checks. One Azure Repos quirk to know: the pr: trigger keyword is ignored for Azure Repos, so PR validation gets wired up through a branch policy build check instead, the way the branch-policies lesson sets it up. Authentication runs on a service connection using workload identity federation (OIDC, OpenID Connect). The pipeline presents a short-lived token that Entra ID has been told to trust, and trades it for temporary Azure credentials. No service-principal password sits anywhere, so there is nothing to expire, leak, or rotate.

trigger:
  branches: { include: [ main ] }
  paths:    { include: [ infra ] }
# Azure Repos: PR runs come from a branch-policy build check, not a pr: block
 
pool: { vmImage: 'ubuntu-latest' }
 
stages:
- stage: Preview
  jobs:
  - job: whatif
    steps:
    - task: AzureCLI@2
      displayName: what-if
      inputs:
        azureSubscription: sc-orders-oidc     # workload identity federation
        scriptType: bash
        scriptLocation: inlineScript
        inlineScript: |
          az deployment group what-if -g rg-orders-prod \
            --template-file infra/main.bicep --parameters infra/prod.bicepparam
 
- stage: Apply
  dependsOn: Preview
  condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
  jobs:
  - deployment: apply
    environment: orders-prod                  # approvals & checks attach HERE
    strategy:
      runOnce:
        deploy:
          steps:
          - checkout: self                    # deployment jobs skip checkout by default
          - task: AzureKeyVault@2
            inputs:
              azureSubscription: sc-orders-oidc
              KeyVaultName: kv-orders-prod
              SecretsFilter: 'Sql--ConnectionString'   # explicit names, never '*'
          - task: AzureCLI@2
            inputs:
              azureSubscription: sc-orders-oidc
              scriptType: bash
              scriptLocation: inlineScript
              inlineScript: |
                az deployment group create -g rg-orders-prod \
                  --template-file infra/main.bicep \
                  --parameters infra/prod.bicepparam \
                  --name "deploy-$(Build.BuildNumber)"

You can queue a run and watch it finish without leaving the terminal, using the Azure DevOps CLI (command line interface) extension:

az pipelines run --name infra-orders --branch main --output table
 
# Run ID    Number       Status      Result    Pipeline ID    Pipeline Name    Source Branch
# --------  -----------  ----------  --------  -------------  ---------------  -------------
# 1847      20260714.3   notStarted            42             infra-orders     main
 
az pipelines runs show --id 1847 --query "{status:status, result:result}"
# {
#   "result": "succeeded",
#   "status": "completed"
# }

Config and secrets out of the repo

The rule is a three-way split. *Code* lives in Git. *Settings* live in Azure App Configuration. *Secrets* live in Azure Key Vault. Identity-based access ties the three together. App Configuration is the settings drawer: it holds plain settings and feature flags, and labels (dev, prod) let one key carry a different value per environment. Flip a flag there and the app picks it up with no redeploy. Key Vault holds anything that grants access: connection strings, API keys, certificates. Create vaults with RBAC authorization (role-based access control) rather than the legacy access-policy model, so vault access follows the same role assignments as everything else in Azure. One naming trick is worth memorizing. Key Vault refuses : in a secret name, and .NET configuration keys are full of colons, so the .NET configuration provider maps -- onto :. The secret Sql--ConnectionString reaches the app as Sql:ConnectionString.

Inside the pipeline, the AzureKeyVault@2 task above pulls the secrets you name into pipeline variables and masks their values in the logs. Masking is a best-effort find-and-replace on the log stream, not encryption. It cannot recognize a secret you base64-encoded or split across two lines, so never transform a secret inside a script that also prints things. And prefer SecretsFilter with explicit names over *. A pipeline that hoovers up every secret in the vault has a blast radius to match.

az keyvault create -g rg-orders-prod -n kv-orders-prod -l westeurope \
  --enable-rbac-authorization true
 
az keyvault secret set --vault-name kv-orders-prod \
  --name Sql--ConnectionString --value "Server=tcp:sql-orders-prod..."
# {
#   "id": "https://kv-orders-prod.vault.azure.net/secrets/Sql--ConnectionString/9f3c2e...",
#   "attributes": { "enabled": true, ... }
# }
 
# the pipeline's identity may READ secrets — nothing more
az role assignment create --role "Key Vault Secrets User" \
  --assignee-object-id $SP_OBJECT_ID --assignee-principal-type ServicePrincipal \
  --scope $(az keyvault show -n kv-orders-prod --query id -o tsv)
 
# settings and feature flags live in App Configuration, labeled per environment
az appconfig kv set -n appcs-orders --key Checkout:TimeoutSeconds \
  --value 30 --label prod --yes
az appconfig feature set -n appcs-orders --feature NewCheckout --label prod --yes
# {
#   "key": ".appconfig.featureflag/NewCheckout",
#   "label": "prod",
#   "state": "off"
# }

Slot swaps: where config meets deployment

An App Service deployment slot is a second, parallel copy of your app (app-orders-prod-staging) with its own code and its own settings. A swap trades what the two slots are running. This is exactly where configuration and deployment collide, because every app setting carries a slotSetting flag. Sticky settings (slotSetting: true) stay glued to the slot they were set on. Everything else travels with the code. Anything that points at an environment (ASPNETCORE_ENVIRONMENT, App Configuration endpoints, connection strings) has to be sticky, or staging's configuration rides the swap straight into production.

The swap itself is choreographed to protect live traffic. Azure copies the production slot's sticky settings onto the staging workers, restarts them, pings the warmup path (WEBSITE_SWAP_WARMUP_PING_PATH, with the responses it will accept listed in WEBSITE_SWAP_WARMUP_PING_STATUSES), and only then flips the routing. Production traffic never lands on a cold or misconfigured worker. A few things never swap at all: custom domains, TLS (transport layer security) bindings, scale rules, and IP restrictions stay bolted to their slot permanently. The two-phase --action preview flow below lets you look at staging *running with production configuration* before you commit to it. The traffic mechanics get their full treatment in the zero-downtime lesson.

az webapp deployment slot create -g rg-orders-prod -n app-orders-prod --slot staging
 
# --slot-settings = STICKY: stays with the slot during a swap
az webapp config appsettings set -g rg-orders-prod -n app-orders-prod --slot staging \
  --slot-settings ASPNETCORE_ENVIRONMENT=Staging \
                  AppConfig__Endpoint=https://appcs-orders-stg.azconfig.io
 
# --settings = travels WITH the code when you swap
az webapp config appsettings set -g rg-orders-prod -n app-orders-prod --slot staging \
  --settings WEBSITE_RUN_FROM_PACKAGE=1
# [
#   { "name": "ASPNETCORE_ENVIRONMENT",   "slotSetting": true,  "value": "Staging" },
#   { "name": "AppConfig__Endpoint",      "slotSetting": true,  "value": "https://..." },
#   { "name": "WEBSITE_RUN_FROM_PACKAGE", "slotSetting": false, "value": "1" }
# ]
 
# phase 1: apply prod's sticky settings to staging, warm it up — no traffic moves
az webapp deployment slot swap -g rg-orders-prod -n app-orders-prod \
  --slot staging --action preview
 
# verify staging now runs with prod config, then complete (or abort):
az webapp deployment slot swap -g rg-orders-prod -n app-orders-prod \
  --slot staging --action swap      # --action reset to roll back phase 1

Drift, blast radius, and the road to release

Run the preview on a schedule, not only on pull requests. terraform plan -detailed-exitcode exits 2 when reality has wandered away from the code, and a nightly pipeline turns that exit code into an alert. That is drift detection with no new tools. (For Bicep, a scheduled what-if that flags any result other than no-change does the same job.) Know the sharp edges as well. The Terraform state file keeps plenty of secret values in plaintext, so the storage account holding it deserves the same care you give Key Vault itself. What-if noise is the reason a human still reads production diffs. And the pipeline's identity is now the single door into your environment, so scope one service connection per environment with least-privilege roles. Anyone who can edit that pipeline can do anything that identity can do.

The quiet hero of the YAML above is one line: environment: orders-prod. Environments are where Azure DevOps hangs approvals, business-hours checks, and exclusive locks, the machinery that decides *when* a reviewed change is allowed to land. That machinery is the next lesson: release pipelines and approvals.

Two different budgets are at play here. PR validation should be cheap and mandatory, so nobody is tempted to route around it: format, lint, IaC scan, what-if or plan, and nothing that takes ten minutes to finish. The apply side should be deliberate and a little slow: protected environment, named reviewers, OIDC authentication, and a change-ticket link if your organization asks for one.

Config and secrets are the other half of the job. Non-secret settings and feature flags go in App Configuration. Anything that grants access goes in Key Vault and gets pulled by name at deploy time. Neither one ever gets echoed into a log. The pipeline identity holds Key Vault Secrets User on the specific vaults it reads from, not Contributor on the whole subscription.

Try this

Add a stage that runs az deployment group what-if or terraform plan on every pull request and posts the summary where reviewers will actually see it. Then gate the apply on main behind an environment approval.

# Example PR validation bits inside azure-pipelines.yml (conceptual):
az deployment group what-if -g rg-lab -f main.bicep -p name=contoso
# or
terraform plan -input=false -no-color
az pipelines environment list -o table

$ az deployment group what-if -g rg-lab -f main.bicep
Resource changes: 1 to create, 0 to modify, 0 to delete.
 
# Sample output
$ az pipelines environment list -o table
Name         Namespace
-----------  ---------
lab-infra
prod-infra

Takeaway

Remember: infrastructure code goes through the same ritual as application code. Scan it, read the what-if or the plan, apply it from CI with a short-lived identity, and pull secrets from Key Vault and App Configuration at deploy time instead of baking them into the template.

Next: wire Checkov or the Microsoft template analyzer into the PR so that public storage accounts and wide-open NSG (network security group) rules fail the build before a human even opens the diff.

Related