Arguments, input & redirection

$1, stdin, pipes, and > vs >>.

A shell script (a text file of commands the shell runs from top to bottom) works like a small machine bolted to three hoses. One hose carries material in. Two carry material out: one for the normal results, and a separate one for complaints when something breaks. Once you know where those hoses connect, you can save a script's output to a file, feed one script into the next, and stop errors from vanishing without a trace. That plumbing is most of what separates a script you can trust in production from one that drops a failure on the floor and still reports success.

Every program on Linux is handed three of these hoses the instant it starts, and each one has a number. Standard input (stdin, the channel data arrives on) is stream 0. Standard output (stdout, where normal results go) is stream 1. Standard error (stderr, where problems go) is stream 2. Keeping results and problems on different streams is the whole trick. It is why a well-behaved tool can stay silent while everything works and speak up only when something goes wrong.

Arguments: the settings you dial in first

Arguments are the values you hand a script when you launch it, before it does a lick of work. They behave like the boxes on a paper form: box one is always the first thing, box two the second, and the order carries the meaning. Bash (the program that reads and runs your commands) refers to them by position. $1 is the first argument, $2 the second, on up the line. $0 is the name of the script itself. $# tells you how many arguments turned up. $@ is all of them together, which is what you reach for when you want to loop over every one.

#!/usr/bin/env bash
echo "script name : $0"
echo "first arg   : $1"
echo "second arg  : $2"
echo "how many    : $#"
echo "all of them : $@"

Notice that $# came back as 3 and $@ held all three words. Those two are how you write a script that bends to however many things it is given, instead of one that only ever copes with a fixed count.

Reading input while the script runs: read

Arguments are set once, at the very start. Sometimes you want to ask a question halfway through and wait for the answer, the way a form pauses on a field it will not let you skip past. The read command does that. It stops, reads one line from standard input, and drops it into a variable you name.

#!/usr/bin/env bash
read -p "What is your name? " name
echo "Hello, $name. Access logged."

The -p flag (short for prompt) prints that message, then read waits for you to type a line and press Enter. Here you typed Nadia. For anything secret, like a password, add the -s flag (short for silent): read -s keeps your keystrokes off the screen, so the secret never ends up visible over your shoulder or sitting in the scrollback.

Redirection: pointing the hoses

By default, a command's results pour straight onto your screen. Redirection re-aims them at a file instead. The mental image is a length of pipe clamped onto the end of a command, with you choosing which bucket it drains into. There are a handful of connectors, and each does one small job.

That is the difference that catches people out: > overwrites the whole file every time, while >> adds to the end. Aim > at a log you cared about and its old contents are gone in an instant. Going the other way, < feeds a file into a command's standard input, so the command reads from that file as though you had typed every line by hand.

The part that matters most for operations work is separating stream 1 from stream 2. Write > to catch the results, 2> to catch only the errors, and &> to catch both in a single file. The report script further down sends its error message to standard error on purpose, which is what lets redirection pull the two streams apart cleanly.

The result file stayed empty and the error dropped into its own file, exactly where you pointed it. There is also a special drain called /dev/null, a bottomless bin built into the system. Anything you redirect into it is thrown away for good, which makes it handy for hushing noise you truly do not care about.

Be careful with that move. Throwing standard error into /dev/null silences the message, not the problem underneath it. A backup that fails, or a scanner that crashes, will look perfectly calm if you piped its errors into the void, and during an incident that missing error is often the exact clue you needed. Silence stderr only when you already know what the noise is, and never on a step whose failure you would actually want to hear about.

Pipes: an assembly line for commands

A pipe, written as the vertical bar |, takes the standard output of one command and shoves it straight into the standard input of the next. No temporary file sits in the middle. Each command does one small job and passes its result down the line, like workers on an assembly line who each bolt on a single part.

Read that left to right. Print the file, pull out the second column (the status word), sort the words so identical ones sit next to each other, then count each run. Four small tools, chained together, turned a list of hosts into a status tally. A lot of real log triage works exactly this way: search for the interesting lines, cut out one field, sort, count.

Here-documents: text you type straight into the script

Sometimes the input you want to feed a command is a few fixed lines, not a file sitting on disk. A here-document (usually shortened to here-doc) is like a sticky note you write by hand and staple to the command: you put the lines right there in the script. You pick a marker word, and everything up to the next line that contains only that marker becomes standard input for the command.

EOF is only the marker most people reach for; any word does the job. Variables like $host expand inside a plain here-doc, so they get filled in before the text reaches the command. Quote the marker as <<'EOF' instead and expansion switches off: the text stays exactly as written, dollar signs and all. That quoted form is the safe pick when you are writing out a config file that legitimately contains dollar signs you do not want the shell touching.

Put it together: a report on any file

Here is a small tool that pulls the pieces into one place. It takes a filename as its first argument and prints a short report. Before it touches anything, it checks that you actually passed a name, and that the file is really there. Both complaints go to standard error, so they never bleed into the real output, and each kind of failure sets its own exit code.

#!/usr/bin/env bash
file="$1"
 
if [ -z "$file" ]; then
  echo "usage: $0 <file>" >&2
  exit 1
fi
 
if [ ! -f "$file" ]; then
  echo "error: no such file: $file" >&2
  exit 2
fi
 
lines=$(wc -l < "$file")
words=$(wc -w < "$file")
bytes=$(wc -c < "$file")
 
echo "report for: $file"
echo "  lines: $lines"
echo "  words: $words"
echo "  bytes: $bytes"

Three things work together here. The >&2 on those echo lines aims the message at standard error, which is why the good run stays clean and the error lines could be split off with 2>. The exit numbers are the script's verdict: 0 means success, and any non-zero number flags a specific kind of failure. $? holds the exit code of the last command, so the next step in an automated pipeline can read it and stop instead of charging ahead on bad data. And $file is wrapped in double quotes everywhere it appears, which matters more than it looks.

Related