Concurrency: threads, processes, asyncio & the GIL

Pick the right model — the GIL, I/O vs CPU bound, and safe subprocess fan-out.

The biggest decision in any Python automation tool is how it does more than one thing at a time, and it turns on a single fact about the language: the standard Python interpreter runs exactly one piece of Python code at any given instant. Pick the wrong model and you pile on locks, race conditions, and complexity for zero speedup. Pick the right one and a fleet scan that took an hour finishes in a minute. For a security or operations job, that gap decides whether a control runs on every deploy or gets quietly disabled because it is too slow.

One key for the whole interpreter

CPython (the reference version of Python you get from python.org or your package manager, written in the C programming language) guards its own internals with a single lock called the Global Interpreter Lock, or GIL. Think of a busy kitchen with one chef's knife that everyone shares. Only the cook holding the knife can chop, and there is exactly one knife. A thread (a strand of execution running inside your program) has to hold the GIL to run Python bytecode (the low-level instructions the interpreter actually executes). One knife means one chopper at a time, no matter how many cooks you put on the line.

Here is the part that makes threads worth having anyway. The moment a thread stops to wait on something outside the interpreter, a network reply, a disk read, a subprocess finishing, it drops the knife so another thread can grab it. A cook waiting on the oven puts the knife down. So for work that is mostly waiting, threads overlap and everything speeds up. For work that is mostly computing, the threads never let go of the knife, they take turns, and you gain nothing. Watch it happen.

Two threads on a multi-core machine, and it got a hair slower. The arithmetic never releases the knife, so the threads queue behind each other and you pay a small tax for the handoffs. Now give the identical work true parallelism by running it in separate processes.

Two processes, each with its own interpreter and its own GIL, run on two cores at once and roughly halve the wall-clock time. That is the entire rule in one experiment. Threads share a knife; processes each bring their own.

Python 3.13 shipped an experimental build with the lock removed, called free-threaded or no-GIL Python, in which threads really can run bytecode in parallel. You can ask any interpreter which mode it is in.

The build named python3.13t (the t is for threaded) reports the lock off. Treat it as a preview, not a plan. Many C extensions you depend on are not yet safe without the GIL, and single-thread code often runs slower on that build. For anything you ship today, assume the lock is there and choose your model around it.

I/O-bound or CPU-bound, that is the fork

Every task you automate falls on one side of a line. Either it spends most of its life waiting for something else, which we call I/O-bound (I/O means input/output, the reading and writing your program does over the network, to disk, or to another process), or it spends most of its life computing, which we call CPU-bound (CPU means central processing unit, the chip that does the actual math). A clerk stuck on hold with the bank is I/O-bound. An accountant grinding through columns of figures is CPU-bound. Timing them tells you which one you have: if the wall-clock time is far longer than the processor time, you are waiting, and you are I/O-bound.

The choice follows directly. I/O-bound work wants concurrency, many operations in flight while the GIL is released on each wait, so reach for threads or asyncio. CPU-bound work wants real parallelism across cores, so reach for processes. Get this backwards and you either add threads that fight over one knife or spin up processes whose pickling overhead (the cost of freezing objects to bytes so they can cross between separate interpreters) swamps the tiny bit of computing you were doing.

The same fan-out pattern covers both bounded cases. concurrent.futures gives you two executors with an identical shape, so switching models is a one-word edit once you know which side of the fork you are on.

from concurrent.futures import (
    ThreadPoolExecutor, ProcessPoolExecutor, as_completed)
import hashlib, requests
 
# I/O-bound: 200 HTTP health checks. Threads win: the GIL is
# released while the socket waits for each reply.
def check(url):
    return url, requests.get(url, timeout=5).status_code
 
with ThreadPoolExecutor(max_workers=32) as ex:
    for fut in as_completed(ex.submit(check, u) for u in urls):
        url, code = fut.result()
        print(code, url)
 
# CPU-bound: SHA-256 a directory tree in pure-Python glue.
# Processes win: real cores, separate GILs.
def digest(path):
    with open(path, "rb") as f:
        return path, hashlib.sha256(f.read()).hexdigest()
 
with ProcessPoolExecutor() as ex:      # defaults to os.cpu_count()
    for fut in as_completed(ex.submit(digest, p) for p in files):
        print(*fut.result())

asyncio when you need thousands in flight

When a run means hundreds or thousands of network calls at once, threads stop scaling. Every operating-system thread costs memory and scheduling overhead, and a few thousand of them bury the machine. asyncio (asynchronous input/output, said out loud as A-sync-I-O) takes a different shape. One thread runs an event loop (a dispatcher that keeps a list of half-finished jobs and resumes whichever one is ready). It works like a single receptionist at a switchboard full of callers on hold, picking up each line the instant it speaks. A coroutine (a function you write with async def that can pause at every await and hand control back) is one caller. The loop juggles them all cooperatively, and because it is all one thread, there is no knife to fight over.

import asyncio, httpx
 
async def probe(client, sem, url):
    async with sem:                       # cap concurrency here
        r = await client.get(url, timeout=10)
        return url, r.status_code
 
async def main(urls):
    sem = asyncio.Semaphore(50)           # at most 50 in flight
    async with httpx.AsyncClient() as client:
        tasks = [probe(client, sem, u) for u in urls]
        return await asyncio.gather(*tasks, return_exceptions=True)
 
results = asyncio.run(main(urls))

Two rules keep this honest. First, cap the concurrency with a Semaphore (a counter that only lets N jobs past at a time, like a bouncer holding a fixed headcount at the door), so you do not open ten thousand sockets and look like a denial-of-service attack (DoS, flooding a service with more requests than it can serve) against your own dependency. Second, everything the loop touches has to be non-blocking. One ordinary blocking call, a synchronous DNS lookup (DNS is the Domain Name System, the phone book that turns a hostname into an address), or a heavy pure-Python loop, freezes every other job on the loop, because the single receptionist is now stuck on one line. Use async-native libraries such as httpx, aiohttp, or asyncpg, or push the blocking work onto a thread or process pool with loop.run_in_executor.

Fanning out subprocesses without opening a door

A huge amount of DevSecOps automation is really one shell command run across many targets: resolve a thousand hostnames, check a TLS (Transport Layer Security, the encryption that puts the S in HTTPS) certificate on each host, grep a config on every box. Running a subprocess (a separate program your script launches and waits on) is I/O-bound from Python's point of view, so a thread pool is the natural fit. The trap is how you hand the command over. Passing one big string with shell=True is like reading a delivery address out loud to someone who will do literally anything the address says. If the address ends with 'and also burn down the house,' they do that too.

#!/usr/bin/env python3
import subprocess, sys
from concurrent.futures import ThreadPoolExecutor, as_completed
 
def resolve(host):
    # List of args, no shell: the host can never become a command.
    r = subprocess.run(
        ["dig", "+short", "+time=2", host],
        capture_output=True, text=True, timeout=5,   # never hang forever
    )
    ips = r.stdout.split() or ["-"]
    return host, ips[0]
 
hosts = [h.strip() for h in sys.stdin if h.strip()]
with ThreadPoolExecutor(max_workers=20) as ex:       # bound the fan-out
    futs = {ex.submit(resolve, h): h for h in hosts}
    for fut in as_completed(futs):
        host, ip = fut.result()
        print(f"{ip:16} {host}")

Now see the door. Swap the list of arguments for one string plus shell=True and hand it a hostname a defender might feed you from an untrusted inventory file.

The dig ran, and then id ran, because the shell saw a semicolon and happily obeyed the second command. That is command injection, and it is the same class of bug behind countless real breaches. Whoever controls the hostname now controls your automation host. As a defender you also want the injection to be visible after the fact: this pattern is exactly why you keep list-form subprocess calls in code review and why you alert on your own tooling spawning an unexpected id, whoami, or curl.

The other half of running commands safely is not drowning yourself. Twenty workers, not two thousand. Unbounded fan-out looks like a flood to whatever you are hitting, trips rate limits, and exhausts your own file descriptors (the small integer handles the kernel hands out for every open file, socket, or pipe). Check your soft cap before you scale up.

Every concurrent socket or subprocess eats one of those 1024 handles, plus a few for pipes. Blow past the cap and Python raises OSError: [Errno 24] Too many open files right in the middle of a run, usually against the most fragile target. Bounding the pool keeps you under the limit and keeps your scan from reading, on the receiving end, as an attack. A capped, predictable fan-out is both faster to reason about and quieter on the network you share with everyone else.

Once you have picked a model, prove it instead of trusting it. Time the run both ways with time.perf_counter(), or wrap the whole thing in /usr/bin/time -v and read the numbers. A CPU-bound job done right pushes total processor use toward 100 percent times your core count and shows wall-clock time close to processor time. An I/O-bound job done right shows low processor use with many connections open at once, wall-clock far below the sum of the waits. If the numbers barely move when you add workers, you are on the wrong side of the fork, and the fix is to switch models, not to add more workers to the model that cannot help.

Try this

Work through “Fanning out subprocesses without opening a door” yourself on a sandbox you can throw away, following the commands above in order. Then break one step deliberately and re-run, so you have seen the failure before it finds you.

Takeaway

The trap worth remembering here: never build a shell string from input you did not write. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related