Running commands safely

subprocess.run with a list, timeouts, errors, option injection and the environment.

Most security automation runs other programs: a scanner, git, grep, a syntax checker. This lesson shows how to do that from Python with subprocess.run so that a hostile file name cannot run commands, a hung tool cannot hang your job, and a child process does not receive secrets it has no use for. You will pass commands as lists, read exit status and output, handle the four ways a call fails, see shell injection and option injection happen in a lab directory, and control the environment and standard input a child gets.

Run a program with a list

subprocess.run() starts a program, waits for it to finish and returns a CompletedProcess object describing what happened. Give it the command as a list: the first item is the program, each further item is exactly one argument.

The lab works in ~/ps-subprocess. Unpack the lesson files in your home directory (or create the scripts as they appear), and run setup.sh first. It creates the small files the examples work on: an app.log with two ERROR lines, a config/ directory with one stored password, a token file and a list of file names another job wrote:

#!/usr/bin/env bash
# Creates the practice files for the subprocess lesson. Safe to run again.
set -euo pipefail
rm -rf config
mkdir config
printf '%s\n' \
  '2026-09-27T09:00:01+00:00 web01 app[812]: INFO service started' \
  '2026-09-27T09:02:14+00:00 web01 app[812]: ERROR database timeout after 5s' \
  '2026-09-27T09:02:15+00:00 web01 app[812]: INFO retrying' \
  '2026-09-27T09:02:20+00:00 web01 app[812]: ERROR database timeout after 5s' >app.log
printf '%s\n' 'print("hello from app.py")' >app.py
printf '%s\n' 'listen=0.0.0.0:8443' 'db_user=app' 'db_password=lab-password-not-secret' >config/app.conf
printf '%s\n' 'listen=0.0.0.0:443' >config/web.conf
printf '%s\n' 'API_TOKEN=lab-token-not-secret' >secrets.env
# Written by "another job" from file names in a shared directory. One name is hostile.
printf '%s\n' 'old-report.csv' '-rf' 'reports' >stale.txt

import subprocess
 
plain = subprocess.run(["grep", "-c", "ERROR", "app.log"])
print("plain:", plain)
 
raw = subprocess.run(["grep", "-c", "ERROR", "app.log"], capture_output=True)
print("bytes:", raw.stdout)
 
result = subprocess.run(
    ["grep", "-c", "ERROR", "app.log"],
    capture_output=True,
    text=True,
    timeout=10,
)
print("text:", repr(result.stdout), "returncode:", result.returncode)

The first call printed 2 by itself: without capture_output=True the child writes straight to your terminal, and the result only records the arguments and the returncode, the exit status that $? shows in Bash. With capture_output=True the output is kept in stdout and stderr, as bytes (b'2\n', the type from the Python basics lesson). text=True decodes them to str with the locale's encoding; for tool output that may contain bad bytes, pass encoding="utf-8", errors="replace" instead, as the files lesson explained. timeout=10 is the limit on how long you will wait. Put a timeout on every call.

Exit status: read returncode, or let check=True raise

run() does not raise an exception when a program fails; it hands you returncode and leaves the decision to you. That is right for tools whose exit codes carry meaning. grep exits 0 when it found a match, 1 when it found none and 2 on an error, and for a secret scan "no match" is the good outcome. This scanner has two parts: grep_hits.py runs grep and reduces its output to locations, and find_secrets.py decides what each outcome means:

"""Run grep for stored passwords and keep only where they are."""
import subprocess
 
# -H: always print the file name, also when the target is a single file
# -Z: end the file name with a NUL byte, so a ":" in a name cannot shift the fields
# -a: read binary files as text; otherwise a match in a file that is not valid UTF-8
#     only prints "binary file matches" on stderr, and stdout stays empty
GREP = ["grep", "-r", "-n", "-H", "-Z", "-a", "-e", "password=", "--"]
 
 
def run_grep(target: str) -> subprocess.CompletedProcess:
    return subprocess.run(
        GREP + [target],
        capture_output=True,
        encoding="utf-8",
        errors="replace",  # a match in a binary file is not valid UTF-8 either
        timeout=30,
    )
 
 
def locations(stdout: str) -> list[str]:
    """FILE:LINE for each match; the matched text (the password) is dropped."""
    hits = []
    for line in stdout.split("\n"):  # not splitlines(): binary data holds other breaks
        name, nul, rest = line.partition("\0")
        if nul:
            hits.append(name + ":" + rest.split(":", 1)[0])
    return hits

import subprocess
import sys
 
from grep_hits import locations, run_grep
 
 
def main(argv: list[str]) -> int:
    if len(argv) != 1:
        print("usage: find_secrets.py PATH", file=sys.stderr)
        return 2
    try:
        result = run_grep(argv[0])
    except subprocess.TimeoutExpired as err:
        print(f"find_secrets: {err}", file=sys.stderr)
        return 2
    hits = locations(result.stdout)
    for hit in hits:
        print(f"password stored in {hit}")
    # grep exits 0 on a match, 1 on none, 2 when something could not be searched.
    # 0 without a parsed hit means grep matched something this script missed.
    if result.returncode not in (0, 1) or (result.returncode == 0 and not hits):
        problem = result.stderr.strip() or "grep matched, but no match could be parsed"
        print(f"find_secrets: incomplete scan: {problem}", file=sys.stderr)
        return 2
    return 1 if hits else 0
 
 
if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))

Each grep status became a distinct result: a finding (exit 1 from the script), a clean file (0), and a typo in the directory name (2, with grep's own message from result.stderr). locations() keeps the file name and line number of each match and drops the matched text, so the report says where the password is without printing it. Two of the grep options exist to keep that promise. Without -H, grep prints file names only when it searches more than one file, so for a single file its output starts with the line number:

Split at the first colon, that line would give 3 as the file name and the password as the line number, printed for everyone to read. -H makes grep print the name in every case. -Z ends the name with a NUL byte, which no Linux path can contain, so a name with a : in it cannot shift the fields; line.partition("\0") splits there.

The second trap is quieter. grep treats a file that is not valid UTF-8 as binary, and a Latin-1 config file, a .jar or an SQLite database all count. For a binary file it prints no line at all:

The message went to stderr, the exit status says "match found", and stdout is empty. A script that trusts the status and parses stdout finds no hit and reports the file as clean. -a makes grep search every file as text, which is the deliberate choice for a secret scan (-I would skip binary files instead, the wrong choice here), and errors="replace" decodes the bytes that are not UTF-8. main() also refuses a status of 0 without a parsed hit, in case grep matches something the parser does not understand:

When a file cannot be read, grep reports it, keeps searching the rest and exits 2. The script prints what it did find and exits 2 as well, so a run that skipped a file never looks complete:

check=True turns any non-zero exit status into a subprocess.CalledProcessError. On grep that treats the clean result as a crash:

Use check=True where non-zero really means failure. bash -n parses a script without running it and exits non-zero on a syntax error, so a deploy gate can let it raise. The exception carries the returncode, the command, and the captured stdout and stderr:

#!/usr/bin/env bash
# Broken on purpose: the if has no "then".
if [ -d /srv/app ]
  echo "deploying"
fi

import subprocess
import sys
 
script = sys.argv[1]
try:
    subprocess.run(
        ["bash", "-n", "--", script],  # -n: parse the script, run nothing
        check=True,
        capture_output=True,
        text=True,
        timeout=10,
    )
except subprocess.CalledProcessError as err:
    print(f"{script}: syntax check failed (exit {err.returncode})", file=sys.stderr)
    print(err.stderr, end="", file=sys.stderr)
    sys.exit(1)
print(f"{script}: syntax OK")

Missing and hung programs

Two more failures never produce an exit status at all. When the program does not exist, run() raises FileNotFoundError before anything starts:

This is the error you get when a tool is missing on one host, or sits in a venv or directory that is not on the job's PATH: cron and systemd never see the PATH your interactive shell built, as the setup lesson showed. Catch it separately from CalledProcessError, or check at start-up with shutil.which(), which returns the full path or None. Calling a tool by its absolute path removes the PATH question entirely.

A tool that hangs is stopped by timeout=. When the time runs out, run() kills the child with SIGKILL, waits for it, and raises TimeoutExpired:

"Kills the child" means exactly one process. With shell=True the child is /bin/sh (dash on Ubuntu), and the programs in a pipeline are its children:

Python gave up after two seconds, but sleep 47 was still running with no parent waiting for it; pkill stopped it by hand, and the second pgrep found nothing. In a job that runs every few minutes, such leftovers pile up. (The brackets in '[s]leep 47' stop pgrep from matching the shell that runs the pgrep command itself, whose command line contains the pattern.) The same happens in list form when the program you start is a script that starts its own children.

The fix is to start the command in its own process group and kill the whole group on timeout. A process group is a set of processes the kernel can signal together, and a child normally joins its parent's group. start_new_session=True starts the child in a new session with a new group whose ID is the child's PID, and everything the child starts joins that group. os.killpg() sends a signal to every process in a group. run() is built on subprocess.Popen, which starts the program and returns at once; the with block closes its pipes and waits for it at the end:

import os
import signal
import subprocess
 
 
def run_with_deadline(cmd: list[str], timeout: float) -> int:
    """Run cmd; on timeout kill it and every process it started."""
    # start_new_session=True puts the child in a new process group whose ID is
    # the child's PID; everything it starts joins that group.
    with subprocess.Popen(cmd, start_new_session=True) as proc:
        try:
            return proc.wait(timeout=timeout)
        except subprocess.TimeoutExpired:
            os.killpg(proc.pid, signal.SIGKILL)  # the whole group, not one PID
            proc.wait()
            raise
 
 
try:
    # A fixed pipeline, run by an explicit shell: no outside input in it.
    run_with_deadline(["sh", "-c", "sleep 48 | cat"], timeout=2)
except subprocess.TimeoutExpired as err:
    print(f"gave up after {err.timeout} s and killed the process group")

pgrep found nothing (exit 1): the timeout took sh, sleep and cat together. Process groups, sessions and signal forwarding in long-running services are covered in scripting-adv.

Untrusted values: shell injection and option injection

With shell=True, run() passes one string to /bin/sh -c, and the shell parses it: ; ends a command, | starts a pipeline, $( ) runs another command. When part of that string comes from outside, the outside decides what runs. This script is unsafe on purpose, run in the lab directory:

import subprocess
import sys
 
name = sys.argv[1]  # comes from outside: a ticket, a web form, a file listing
# UNSAFE: the value is pasted into a command line that /bin/sh parses
subprocess.run(f"wc -l {name}", shell=True)

wc printed 1 app.py, and secrets.env is gone. The shell read the ; as the end of the wc command and ran rm -f secrets.env as a second one. This is command injection (CWE-78). Here is the list form, after restoring the file:

import subprocess
import sys
 
name = sys.argv[1]
result = subprocess.run(
    ["wc", "-l", "--", name],  # one list item = one argument, no shell
    capture_output=True,
    text=True,
    timeout=10,
)
print(f"exit {result.returncode}:", result.stdout or result.stderr, end="")

There is no shell, so the whole value reached wc as one argument: a file name that does not exist, exit 1, and secrets.env untouched. A normal name works as before.

shell=True is acceptable when the whole string is a fixed literal in your code and you need a shell feature, such as "ps -eo comm | sort | uniq -c". If outside data must reach a shell, shlex.quote() quotes one value for a POSIX shell, but passing a list and doing the rest in Python is simpler to get right. Static analysers such as Bandit flag shell=True; the scripting-adv release-gates lesson adds one to CI.

The list form still has a gap. A value that starts with - is read as an option by the program you start, just as a file named -rf was in the Bash quoting lesson. purge.py deletes the files named in stale.txt, which another job writes from names in a shared directory:

import subprocess
from pathlib import Path
 
# stale.txt is written by another job from names in a shared directory.
names = Path("stale.txt").read_text(encoding="utf-8").split()
subprocess.run(["rm", "-v"] + names, timeout=30)

rm without -r refuses to delete a directory, which is the safety net the author relied on. The entry -rf became options, and rm deleted reports/ with everything in it. This is argument injection (CWE-88). -- tells a program that everything after it is an operand, never an option:

With --, -rf was a file name that did not exist, reports was refused as a directory, and q3.csv survived. An option does not have to be destructive to cause harm:

wc printed its version and exited 0, so a caller that trusts the status believes it counted something. Not every program accepts --, so also validate values from outside: refuse a name that starts with -, or pass it as ./name. And -- answers only "is this an option?", not "may this be deleted?": a line in stale.txt naming ../ps-files/report.txt or a file in ~/.ssh would still be removed, and .split() cuts a name with a space into two names. A real purge job reads one name per line with splitlines(), resolves each one and checks that it stays inside the shared directory, as the files lesson's safe_path() does, and checks returncode.

What the child inherits: environment and stdin

A child gets a copy of your whole environment unless you say otherwise. If your job holds an API token in an environment variable, every scanner, archiver and helper it starts receives the token too:

import os
import subprocess
 
print("parent has API_TOKEN:", "API_TOKEN" in os.environ)
 
# printenv exits 0 when the variable is set in its environment, 1 when not.
inherited = subprocess.run(["printenv", "API_TOKEN"], capture_output=True, timeout=10)
print("child with the inherited environment: exit", inherited.returncode)
 
minimal_env = {"PATH": "/usr/bin:/bin", "LANG": "C.UTF-8"}
scrubbed = subprocess.run(
    ["printenv", "API_TOKEN"], capture_output=True, timeout=10, env=minimal_env
)
print("child with env=minimal_env:          exit", scrubbed.returncode)

printenv found API_TOKEN in the inherited environment (exit 0) and nothing with env=minimal_env (exit 1). env= replaces the environment completely, so give the child what it needs: PATH, which run() also uses to find the program, and LANG for its messages. Build the dictionary explicitly; copying os.environ and deleting the names you know about misses the ones you do not.

When a child does need a secret, pass it on standard input with input=, not as an argument. Arguments are readable by every user on the host through ps and /proc/PID/cmdline while the program runs; the secrets lesson demonstrates that leak.

import os
import subprocess
 
token = os.environ["API_TOKEN"]
# The token reaches sha256sum through a pipe on its standard input. As an
# argument it would be readable by every user on the host in ps output.
result = subprocess.run(
    ["sha256sum"],
    input=token,
    capture_output=True,
    text=True,
    timeout=10,
    env={"PATH": "/usr/bin:/bin", "LANG": "C.UTF-8"},
)
print("token fingerprint:", result.stdout.split()[0][:16])

sha256sum read the token from the pipe, and the fingerprint matches what the same shell pipeline computes, so the value arrived intact. A plain hash identifies a long random token without revealing it. It does not protect a password or a short PIN: anyone can hash guesses until one matches, as the crypto lesson shows, so never log a hash of those. With text=True, input= takes a str; without it, bytes.

Try this

Write count_matches.py PATTERN FILE... in ~/ps-subprocess. It runs grep -c -H -e PATTERN -- FILE... as a list with timeout=10 and an env= holding only PATH and LANG, prints grep's output, sends its errors to stderr, and exits with grep's status (0 match, 1 none, 2 error); a missing grep or a timeout prints one line and exits 2. Break it first: build the command as an f-string with shell=True, run it with the pattern 'ERROR app.log; touch pwned #' and app.log, and confirm that the output looks normal (app.log:2) while a file named pwned appeared. Then fix and verify: ERROR app.log config/app.conf prints app.log:2 and config/app.conf:0 with exit 0; the injection pattern prints app.log:0, exits 1, and creates no pwned; ERROR -rf reports grep: -rf: No such file or directory and exits 2.

Takeaway

Pass commands as lists with -- before operands from outside, set a timeout on every call, decide from returncode or check=True which exit codes are failures, and give each child only the environment it needs, with secrets on standard input.

Related