Bash internals: subshells, FDs & process substitution

Subshells vs the current shell, file descriptors, process substitution and coprocesses.

Most "advanced Bash" mistakes come from not knowing when the shell forks a new process and when it runs code in the current one. A subshell has its own copy of every variable, its own working directory, and its own exit status; changes inside it evaporate when it ends. Knowing exactly where subshells appear — every pipe stage, every $(...), every ( ... ), every backgrounded job — is the difference between a script that behaves and one that mysteriously loses state.

Subshells vs the current shell

A parenthesised group ( ... ) runs in a subshell; a brace group { ...; } runs in the current shell. This matters the moment you assign a variable: a value set inside a subshell is invisible outside it. The classic trap is the pipeline — every command in a | b | c runs in its own subshell, so a variable set in the last stage of a pipe does not survive. This is why while read ... done < file works but cmd | while read ... loses its counter.

# BROKEN: count is set in a subshell (right side of the pipe) and lost
count=0
printf 'a\nb\nc\n' | while read -r line; do count=$((count+1)); done
echo "$count"      # -> 0  (the while ran in a subshell)
 
# FIX 1: redirect instead of pipe, so while runs in the current shell
count=0
while read -r line; do count=$((count+1)); done < <(printf 'a\nb\nc\n')
echo "$count"      # -> 3
 
# FIX 2 (bash 4.2+): shopt -s lastpipe runs the last pipe stage in-shell
shopt -s lastpipe; set +m
printf 'a\nb\nc\n' | while read -r line; do count=$((count+1)); done
echo "$count"      # -> 3

File descriptors are the real interface

Redirection is not magic syntax — it is manipulation of the process file-descriptor table. 0 is stdin, 1 is stdout, 2 is stderr, and you can open more. 2>&1 means "make FD 2 point at whatever FD 1 currently points at" — order matters, because it copies the current target. You can open your own descriptors for logging, keep the original stdout aside while you redirect, and restore it later. Treating FDs as first-class lets you build clean logging without temp files.

# order matters: this sends BOTH to the file...
cmd >file 2>&1
# ...this sends stdout to file, but stderr to the OLD stdout (the terminal)
cmd 2>&1 >file
 
# open FD 3 as a log sink; save real stdout in FD 4; restore later
exec 3>>/var/log/tool.log
exec 4>&1            # stash current stdout
exec 1>&3            # redirect stdout to the log
echo "this goes to the log"
exec 1>&4 4>&-       # restore stdout, close the stash
echo "this goes to the terminal again"

Process substitution and coprocesses

Process substitution, <(cmd) and >(cmd), exposes a command’s stdout or stdin as a filesystem path (usually /dev/fd/NN). It lets you feed the output of a command to a program that insists on a filename, or tee into several consumers, all without temporary files and their attendant race conditions. A coprocess (coproc) goes further, running a background job with its stdin and stdout wired to FDs you can talk to — useful for driving a long-lived helper.

# compare two commands’ output directly
diff <(kubectl get cm -o yaml) <(kubectl get cm -n staging -o yaml)
 
# tee a stream into several processors at once
generate_report | tee >(gzip >report.gz) >(sha256sum >report.sha) >/dev/null
 
# feed a process-substituted file to a tool that demands a path
openssl dgst -sha256 <(curl -fsSL https://example.com/artifact)