Running commands safely with subprocess

Argument lists, never shell=True with f-strings.

Automation is mostly glue. Your Python script rarely does the security work itself. It runs the tools that already do it. It calls git to read a commit, ls to see what shipped, a secret scanner to check a diff, then reads what each tool printed and decides what to do next. The module that lets Python start another program is subprocess (short for sub-process, meaning a second program your script launches and then waits on). Learn one function from it well and you can drive almost any command-line tool. Learn it carelessly and you hand an attacker a way to run their own commands on your build server.

The one call to learn: subprocess.run

subprocess.run() starts a program, waits for it to finish, and hands back an object that describes what happened. The habit that keeps you safe is how you pass the command. Give it a list of separate strings, one word per item, never one long sentence. Think of a paper form with labelled boxes instead of a handwritten note. The list ["git", "rev-parse", "HEAD"] says the program is git, its first argument is rev-parse, its second is HEAD. You fill every box yourself, and nothing that arrives later can slip in a box of its own. Hold onto that picture. It is the whole security lesson in one sentence.

# commit.py
import subprocess
 
result = subprocess.run(
    ["git", "rev-parse", "HEAD"],
    capture_output=True,   # keep the output instead of printing it
    text=True,             # give me strings, not raw bytes
    check=True,            # raise if git reports failure
    timeout=10,            # give up after 10 seconds
)
 
print("exit code:", result.returncode)
print("commit:", result.stdout.strip())

Four keyword arguments earn their place on nearly every call. capture_output=True keeps the program's output for you instead of letting it spill onto your screen. text=True hands you that output as ordinary strings; without it you get raw bytes you would have to decode yourself. check=True raises an exception the moment the program reports failure, so a broken command stops your script loudly instead of sliding past unnoticed. timeout=10 gives the command ten seconds, then gives up.

Reading the result

subprocess.run gives you three things worth reading, and it helps to picture the finished command handing you two trays and a grade. returncode is the grade: the exit code, a single number every command returns when it ends. Zero means success, by a Unix convention older than most of us; anything else signals some kind of failure. stdout, short for standard output, is the first tray, the normal text a program prints. stderr, short for standard error, is the second tray, a separate channel programs use for warnings and errors so they never get mixed into the real results. That separation is why a stray warning line can never corrupt the commit hash you were trying to capture.

When a command fails

With check=True, a non-zero exit code turns into a CalledProcessError you can catch. The exception carries the same returncode and stderr, so you can log exactly what the tool complained about and react instead of guessing.

# check_ref.py
import subprocess
 
try:
    subprocess.run(
        ["git", "rev-parse", "--verify", "badref"],
        capture_output=True, text=True, check=True,
    )
except subprocess.CalledProcessError as e:
    print("git failed with code", e.returncode)
    print("stderr:", e.stderr.strip())

timeout behaves the same way. If the command runs past its limit, Python raises subprocess.TimeoutExpired and stops the child process for you. That one line is what stands between a healthy pipeline and a scanner that hangs on a dead host and never returns. Remember that subprocess.run() sets no time limit of its own. Leave timeout off and a program that freezes will freeze your script right along with it. Any command that touches the network or reads input you did not choose deserves an explicit timeout= and a caught subprocess.TimeoutExpired.

The trap: shell=True turns a filename into a weapon

There is a second way to call subprocess.run: hand it one string and add shell=True. Python then starts a real shell (/bin/sh on Linux, the same command interpreter your terminal uses) and asks it to read your string, exactly the way your terminal would. The shell is eager and powerful. It expands *, follows pipes written with |, and, most dangerously, treats a semicolon ; as "end of one command, start of the next." That last feature is where automation gets breached.

Say you build the command with an f-string (Python's format-string syntax, where {name} drops a variable straight into the surrounding text) and the value comes from somewhere you do not control: a filename, a form field, a webhook payload.

# danger.py  --  DO NOT SHIP THIS
import subprocess
 
# This value came from a user; you did not choose it.
untrusted = "app.py; rm -f secrets.env"
 
subprocess.run(f"wc -l {untrusted}", shell=True)

You meant to count the lines in one file. Watch what the shell actually does with it. The ls before and after show the damage.

The shell saw two commands split by that semicolon. It ran wc -l app.py, printed 1 app.py, then obediently ran rm -f secrets.env and deleted your secrets file. Your Python never intended a second command; the shell invented one from the input. This is command injection, catalogued as CWE-78 (Common Weakness Enumeration entry 78, the industry's catalog number for operating-system command injection), one of the oldest and most damaging bugs there is.

The argument list is immune

Switch to the list form and the danger evaporates, because no shell is involved. The operating system starts the program you named and passes the rest as arguments, one item as exactly one argument. Nothing is present to read the semicolon, so the whole hostile string becomes a single, harmless filename that happens not to exist.

# safe.py
import subprocess
 
untrusted = "app.py; rm -f secrets.env"
 
result = subprocess.run(["wc", "-l", untrusted], capture_output=True, text=True)
print("exit code:", result.returncode)
print(result.stderr.strip())

wc looked for a file literally named app.py; rm -f secrets.env, failed to find it, and returned a non-zero exit code. secrets.env is untouched. Same input, same tool, opposite outcome, and the only change was passing a list instead of a string with shell=True.

When shell=True is actually fine

shell=True is not forbidden. It is fine when the entire command string is a fixed literal you typed yourself, with no outside value spliced in, and you genuinely need a shell feature the list form cannot give you: a pipe, a glob, a redirect, or a shell builtin. subprocess.run("git log --oneline | head -5", shell=True) is acceptable because every character came from you. The rule is not "never use a shell." The rule is "never let untrusted data reach one." If you are ever forced to combine the two, wrap each value with shlex.quote(), which seals a hostile string inside safe quoting so the shell reads it as one token. Even then, reaching for the list form and doing the pipe in Python is safer and usually clearer.

Putting it together: a scanner gate

Here is the payoff. A DevSecOps job that comes up constantly is failing a build when a scanner finds something. bandit is a SAST tool (Static Application Security Testing, meaning it reads your source code without running it and flags risky patterns). Point it at the very mistake from earlier.

# deploy.py
import subprocess
 
def run_backup(filename):
    subprocess.run(f"tar -czf backup.tgz {filename}", shell=True)

The scanner caught the same shell=True footgun and labelled it CWE-78, High severity. Now wrap the scanner in Python so a pipeline can act on it. Run bandit with an argument list, read its exit code, and turn a finding into a failed build.

# gate.py: fail the build when the scanner finds a High-severity issue
import subprocess, sys
 
result = subprocess.run(
    ["bandit", "-q", "-r", "deploy.py"],
    capture_output=True, text=True,
)
 
if result.returncode == 0:
    print("scan clean")
    sys.exit(0)
 
highs = result.stdout.count("Severity: High")
print(f"scan failed: {highs} high-severity issue(s)")
sys.exit(1)

bandit exits non-zero when it finds problems, so gate.py exits non-zero too, and any CI system (Continuous Integration, the service that runs your checks automatically on every push) reads that as a red build. That is the shape of almost every automation you will write: run a tool with a list, read its exit code and output, decide, and pass a clear signal up the chain. Get the list-versus-string habit right and every one of those scripts is safe by construction.

Related