CoursesAdvanced scripting for DevSecOpsRobust CLIs: click, typing, dataclasses & packaging

Robust CLIs: click, typing, dataclasses & packaging

Structured tools with subcommands, type hints, config and clean exit codes.

Advanced35 min · lesson 8 of 15

From Sticky Note To Control Panel

A one-off script is a sticky note. You wrote it for yourself, you know its quirks, and when it breaks you shrug and patch it. The moment two coworkers start running it inside a pipeline at three in the morning, it stops being a note and becomes a control panel, and a control panel has to be obvious to operate and hard to press the wrong button on. This lesson is about that jump: taking a script that works and turning it into a tool that is clear to run, hard to misuse, and honest about whether it actually succeeded.

For a security or operations team, that last word carries the most weight. A tool that rotates a credential, revokes a token, or deletes a firewall rule acts with real blast radius. You want named subcommands so each action has a name, typed flags so a fat-fingered value gets rejected at the door, a dry run for anything destructive, and an exit code that tells the truth so the automation around it can trust the result.

Subcommands And Typed Flags

Reading sys.argv (the raw list of words from the command line, your program's name followed by everything the user typed after it) by hand is like sorting the mail by squinting at each envelope. You will get it mostly right, and you will be wrong in ways you never notice. Python ships with argparse (the argument parser in the standard library), which already handles subcommands, converts strings to the types you ask for, and prints a generated --help. click and typer are third-party libraries that wrap the same work in decorators and hand back friendlier error messages. Pick one and let it own the parsing.

The shape that scales is a verb per action and a noun for what it acts on, grouped under a single top-level command. Give every flag a type and a default. Attach --dry-run to anything that changes the world, and make the dry run print exactly what the real run would do, then stop.

sec_tools/cli.py
# sec_tools/cli.py
import sys
import click
from .config import load_config
from .rotate import do_rotate
@click.group()
@click.option("-v", "--verbose", count=True, help="Repeat for more detail.")
@click.pass_context
def cli(ctx: click.Context, verbose: int) -> None:
ctx.obj = {"verbose": verbose}
@cli.command()
@click.argument("target")
@click.option("--endpoint", help="Vault URL; overrides APP_ENDPOINT.")
@click.option("--dry-run", is_flag=True, help="Show what would happen, change nothing.")
@click.pass_context
def rotate(ctx: click.Context, target: str, endpoint: str | None, dry_run: bool) -> None:
"""Rotate credentials for TARGET."""
cfg = load_config(endpoint)
if dry_run:
click.echo(f"[dry-run] would rotate {target} via {cfg.endpoint}")
return
sys.exit(do_rotate(cfg, target))
@cli.command()
def audit() -> None:
"""Report which credentials are past their rotation window."""
...
if __name__ == "__main__":
cli()
~/secopslog — bash
$ sec-rotate --help
Usage: sec-rotate [OPTIONS] COMMAND [ARGS]... Options: -v, --verbose Repeat for more detail. --help Show this message and exit. Commands: audit Report which credentials are past their rotation window. rotate Rotate credentials for TARGET.
$ sec-rotate rotate db-prod --endpoint https://vault.internal:8200 --dry-run
[dry-run] would rotate db-prod via https://vault.internal:8200

Two useful things fall out of that for free. sec-rotate --help lists every subcommand without you writing a line of help text, so an operator who has never touched the tool can discover what it does. And --dry-run lets that same operator rehearse a destructive action against production and read back the endpoint the tool resolved before anything moves.

Types And Dataclasses That Fail Loud

Type hints are labels on the jars in your pantry. Python does not check them while the program runs, but a type checker reads the labels before you cook and warns you when you are about to pour salt where the recipe asked for sugar. That checker is mypy. Wire it into your CI (continuous integration, the automated system that builds and tests every change) and a whole category of wrong-shape bugs never reaches a real server.

A dataclass (a small class where Python writes the repetitive parts for you) is a typed record with almost no boilerplate. It is the right home for configuration and for results, because every field has a name and a type. Mark it frozen=True and nobody can quietly mutate it after you have validated it, which is exactly what you want for settings that decide where a destructive command points.

sec_tools/config.py
# sec_tools/config.py
import os
from dataclasses import dataclass
import click
from .exit_codes import Exit
@dataclass(frozen=True)
class Config:
endpoint: str
timeout: float = 10.0
concurrency: int = 16
def load_config(flag_endpoint: str | None) -> Config:
# precedence: explicit flag > environment variable > built-in default
endpoint = flag_endpoint or os.environ.get("APP_ENDPOINT")
if not endpoint:
click.echo("error: set --endpoint or APP_ENDPOINT", err=True)
raise SystemExit(Exit.CONFIG)
return Config(
endpoint=endpoint,
timeout=float(os.environ.get("APP_TIMEOUT", "10")),
)

Introduce a bug on purpose. Hand timeout the raw environment string instead of wrapping it in float(). The program might run for weeks until some comparison behaves oddly under load. Run mypy first and the mistake never survives the commit.

~/secopslog — bash
$ mypy sec_tools/
sec_tools/config.py:25: error: Argument "timeout" to "Config" has incompatible type "str"; expected "float" [arg-type] Found 1 error in 1 file (checked 5 source files)

mypy names the file, the line, and the exact type that does not fit, so you fix it while the context is still in your head instead of reading a production stack trace at midnight.

Config With A Clear Order Of Precedence

When a value can arrive from several places, you need one rule for who wins. Think of a ring of keys with a pecking order. The key you hand over in person (the command-line flag) opens the lock even though there is a spare under the mat (an environment variable, a setting the shell holds in memory for every program it launches) and a master copy back at the office (a config file on disk). Explicit beats ambient. The order almost every good tool follows, highest priority first: command-line flag, then environment variable, then config file, then the built-in default.

Resolve all of it once, at startup, into that frozen Config, and validate it right there. From then on the rest of the program trusts its inputs and never reaches back into the environment. For a security tool that is a guard rail. You choose the endpoint a destructive action points at in exactly one place, so a stray environment variable left over from an earlier session cannot silently redirect a rotation at the wrong vault (the service that holds your secrets).

~/secopslog — bash
$ export APP_ENDPOINT=https://vault.internal:8200 sec-rotate rotate db-prod --endpoint https://vault.staging:8200 --dry-run
[dry-run] would rotate db-prod via https://vault.staging:8200

The flag wins even though APP_ENDPOINT is set, which is the behavior you want: what the operator typed on this exact line beats whatever the environment happened to be carrying from before.

Exit Codes Are A Contract With The Shell

Every command leaves a number behind when it finishes, its exit code, the way a delivery driver leaves a slip on the door: delivered, nobody home, wrong address. The shell stores that number in the variable $? (the shell variable holding the exit code of the last command) and your pipeline reads it to decide what happens next. Zero means success. Any non-zero means failure, and if you give distinct failures distinct numbers, the caller can react: retry a network blip, page a human on a rejected credential, halt the pipeline on a bad config.

Name the codes so they are not magic numbers scattered through the source. An IntEnum (an enumeration whose members double as plain integers) reads clearly in the code and still behaves like the number the shell expects. Send the human-readable error to stderr (standard error, the stream meant for messages) and keep it out of stdout (standard output, the stream that carries the tool's real data), so a caller piping the output still gets clean results.

sec_tools/exit_codes.py
# sec_tools/exit_codes.py
from enum import IntEnum
class Exit(IntEnum):
OK = 0
CONFIG = 2 # bad or missing configuration
AUTH = 3 # target rejected the credential
UNREACHABLE = 4 # target down or network failure
~/secopslog — bash
$ sec-rotate rotate db-prod --endpoint https://vault.internal:8200 echo "exit=$?"
error: vault.internal:8200 unreachable (connection refused) exit=4

A wrapper watching this job can now branch on the number instead of scraping log text. Exit 4 means the target was unreachable, so retry in five minutes. Exit 3 means the target rejected the credential, which is a human problem, so page someone. Exit 2 means the config was wrong before any change was attempted, so stop and fix the invocation. Each number is a different story, and the caller never has to guess.

A green run that quietly failed
A top-level except Exception: that logs the error and then falls through to a normal finish turns every failure into a success as far as the shell can tell. The rotation dies, the old credential stays valid, and the pipeline glows green while a stale secret sits in production for an attacker to reuse. Let unexpected exceptions propagate (Python already exits non-zero and prints a traceback), or catch a specific exception and exit with a deliberate non-zero code. Swallowing errors to keep the output tidy is how a broken job reports success.

Package It So People Can Install It

A script you run as python /opt/scripts/rotate.py is a loose part rattling in a drawer. Nobody can tell you its version, which libraries it depends on, or whether the copy on this host matches the copy on that one. Packaging turns it into a labeled tool on a shelf. pip install (pip is the Python package installer) drops a real command onto the user's PATH (the list of directories the shell searches when you type a command) and records the exact libraries the tool needs.

The recipe lives in pyproject.toml (the standard file that describes a Python project). You declare the name, the version, the lowest Python it supports, the dependencies with version bounds, and a console-script entry point that ties a command name to a function inside your package.

pyproject.toml
# pyproject.toml
[project]
name = "sec-tools"
version = "1.2.0"
requires-python = ">=3.10"
dependencies = ["click>=8", "httpx>=0.27"]
[project.scripts]
sec-rotate = "sec_tools.cli:cli" # installs a `sec-rotate` command
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

Build it into a virtual environment (an isolated folder of packages that keeps this tool's dependencies off the system Python), and pip drops the command straight onto your PATH.

~/secopslog — bash
$ python3 -m venv .venv source .venv/bin/activate pip install -e . which sec-rotate
Obtaining file:///home/ops/sec-tools Installing build dependencies ... done Checking if build backend supports build_editable ... done Getting requirements to build editable ... done Preparing editable metadata (pyproject.toml) ... done Collecting click>=8 (from sec-tools==1.2.0) Using cached click-8.1.7-py3-none-any.whl (97 kB) Collecting httpx>=0.27 (from sec-tools==1.2.0) Using cached httpx-0.27.2-py3-none-any.whl (76 kB) Collecting anyio (from httpx>=0.27->sec-tools==1.2.0) Using cached anyio-4.4.0-py3-none-any.whl (86 kB) Collecting certifi (from httpx>=0.27->sec-tools==1.2.0) Using cached certifi-2024.8.30-py3-none-any.whl (167 kB) Collecting httpcore==1.* (from httpx>=0.27->sec-tools==1.2.0) Using cached httpcore-1.0.5-py3-none-any.whl (77 kB) Collecting idna (from httpx>=0.27->sec-tools==1.2.0) Using cached idna-3.8-py3-none-any.whl (66 kB) Collecting sniffio (from httpx>=0.27->sec-tools==1.2.0) Using cached sniffio-1.3.1-py3-none-any.whl (10 kB) Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx>=0.27->sec-tools==1.2.0) Using cached h11-0.14.0-py3-none-any.whl (58 kB) Building wheels for collected packages: sec-tools Building editable for sec-tools (pyproject.toml) ... done Created wheel for sec-tools: filename=sec_tools-1.2.0-0.editable-py3-none-any.whl size=2811 sha256=9b1f0a3c2d4e5f60718293a4b5c6d7e8f9012a3b4c5d6e7f8091a2b3c4d5e6f70 Successfully built sec-tools Installing collected packages: sniffio, idna, h11, certifi, click, httpcore, anyio, httpx, sec-tools Successfully installed anyio-4.4.0 certifi-2024.8.30 click-8.1.7 h11-0.14.0 httpcore-1.0.5 httpx-0.27.2 idna-3.8 sec-tools-1.2.0 sniffio-1.3.1 /home/ops/sec-tools/.venv/bin/sec-rotate

Now sec-rotate is a command on the PATH, installed from a versioned package with pinned dependencies. That is auditable: you can point at the exact code and the exact library versions running in production, which is the line between a shared tool and a snippet someone pasted into a wiki.

One run of a well-behaved CLI
1Parse the command
click or argparse turn argv into a typed subcommand
2Load and validate config once
flag > env > file > default, frozen at startup
3Run the subcommand
--dry-run guards anything destructive
4Return an exit code
0 for success, a distinct number per failure class
5Shell and CI branch on $?
retry, page, or stop; green only when it truly worked
Quick check
01Your nightly credential-rotation job wraps its whole body in try: ... except Exception: log.error(e) with no re-raise and no sys.exit. Tonight the rotation fails because the vault is down. What does your CI dashboard show, and why does it matter for security?
Incorrect — the except block swallows the exception, so the process finishes normally and exits 0.
Correct — a swallowed exception plus a normal exit reports success even though the security action never happened.
Incorrect — catching an exception stops it there; there is no automatic re-raise.
Incorrect — CI branches on the process exit code, not on whatever the logs happen to say.
02The lesson calls type hints 'labels on the jars.' What does that imply about when the hints are checked?
Incorrect — CPython does not check annotations while running; a wrong type flows through untouched.
Correct — hints are inert at runtime and only bite when a tool like mypy reads them ahead of time.
Incorrect — a type checker very much acts on them, catching the str-where-a-float-belongs bug the lesson demonstrates.
Incorrect — frozen=True blocks mutation after creation; it does not validate field types.
03APP_ENDPOINT is exported as https://vault.internal:8200 in the environment, and the operator runs sec-rotate rotate db-prod --endpoint https://vault.staging:8200 --dry-run. Which endpoint does the tool resolve, and why?
Incorrect — the lesson's precedence puts the explicit flag above the environment variable, not below it.
Incorrect — having both set is normal; the precedence rule resolves it without an error.
Incorrect — precedence is deterministic and fixed, not dependent on read order.
Correct — explicit beats ambient, so what the operator typed on this exact line wins over a leftover environment variable.

One check before you ship: run the tool against a deliberately broken endpoint and read echo $?. If it prints a frightening traceback but leaves the exit code at 0, every piece of automation downstream will trust a lie and mark the run green. Make the number match the truth, and only then hand the tool to the team.

Try this

Work through “Package It So People Can Install It” yourself on a sandbox you can throw away, following the commands above in order. Then break one step deliberately and re-run, so you have seen the failure before it finds you.

Takeaway

The trap worth remembering here: a green run that quietly failed. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related