Quoting, wildcards, history and aliases

How the shell expands what you type.

Before bash runs a command, it rewrites the line you typed: it replaces variables and wildcards, splits the result into words and removes your quotes. Most surprises at the prompt, and most bugs in shell scripts, come from not knowing what that rewrite produced. This lesson shows the steps one at a time, so you can predict what a command will receive, quote names with spaces correctly and see what a wildcard matches before you delete anything. It finishes with the two conveniences that live in your own shell, the command history and aliases.

Quoting: what the shell leaves alone

The examples use two practice directories: ~/tips/logs, holding a set of empty files with names chosen to show how wildcards match (including a hidden .env), and ~/tips/dash, used near the end. Create them first:

Spaces separate the words of a command line, so a file name that contains a space arrives as two arguments unless you quote it:

ls was asked for two files, q1 and report.txt, and found neither. There are three ways to keep the name in one piece: double quotes, single quotes, or a backslash before the space, which takes away the special meaning of the next character.

All three name the same file. The difference between the two kinds of quotes is what happens to $. Inside double quotes the shell still replaces variables and $(...); inside single quotes nothing at all is replaced:

Use single quotes for text that must reach a program exactly as written, such as a grep pattern or an awk program, and double quotes when you want a variable's value inside. The rule that matters most concerns variables. file="..." sets a shell variable (the next lesson covers variables), and printf "[%s]\n" prints each argument it receives in brackets on its own line, which shows exactly how the shell split the words:

Unquoted, the variable's value was split at the space into two arguments; quoted, it stayed one. With rm $file that difference deletes the wrong file. Write "$file" every time you use a variable that holds a path.

Wildcards

Wildcards, also called globs, describe file names by pattern: * matches any run of characters, including none, ? exactly one character, and [ct] one character from the set. The shell replaces the pattern with the matching names, sorted, before the command starts (names in the current directory, unless the pattern includes a path such as /etc/*.conf), so the command never sees the pattern. echo prints its arguments, which makes it the way to see what a pattern becomes:

log-?.txt did not match log-10.txt, because ? stands for exactly one character. The practice directory also holds .env, a hidden file of the kind that often contains secrets, and * does not match names that start with a dot:

ls -A lists the dotfile (one name per line, because its output here is not a terminal); * skipped it. So cp * /backup/ leaves .env behind, and a cleanup with rm * leaves it in place.

When nothing matches, bash does not report an error. It passes the pattern on unchanged, and the command then looks for a file literally called *.bak:

Because the pattern is expanded before the command runs, you can check it first. Put echo in front of a command that uses a wildcard and read the list, then run it without echo:

One more trap: a file name that starts with a dash looks like an option once the wildcard has expanded. This directory holds notes.txt and a file called -l:

ls * became ls -l notes.txt, so ls took -l as its long-format option and never listed that file. -- marks the end of the options, and everything after it is treated as a name; ./* works as well, because every match then starts with ./. With rm, a file named -rf in the wrong directory turns rm * into a recursive delete, so write rm -- *.log or rm ./*.log, preview with echo first, and join a cd and a delete with &&, never ;, as the lesson on pipes and exit status showed.

Brace expansion and command substitution

Braces generate words without looking at the disk: a comma-separated list, or a range such as 01..03. An empty item before the comma is allowed, which gives a common shortcut for backups:

cp app.log{,.bak} expands to cp app.log app.log.bak. $(COMMAND) runs a command and puts its output into the line in its place, with the trailing newline removed:

date +%F prints the date as year-month-day, so the copy carries today's date in its name. Quote a substitution whose output could contain spaces, "$(...)", for the same reason you quote variables. Older scripts put the command between backticks instead; $(...) is the modern form because it can be nested and is easier to read.

History and reverse search

bash remembers the commands you type in an interactive session and writes them to ~/.bash_history when the shell exits. The keys that use it work at a prompt, not in a script: the Up and Down arrows step through earlier commands; Ctrl-R starts a reverse search, where you type any part of an old command and press Ctrl-R again for older matches, Enter to run the match or an arrow key to edit it first, and Ctrl-G to give up. history lists the remembered commands with numbers, !! repeats the last command (sudo !! reruns it with sudo), and !$ stands for the last argument of the previous command.

Ubuntu's default ~/.bashrc decides what is remembered:

ignoreboth means commands that start with a space are not saved, and a command identical to the one before it is saved once. In an interactive session on the lab server, deploy typed cd /var/log, then echo not-in-history with a leading space, then ls -d /var/log twice, then exit. The history file afterwards:

RHEL is different. Its /etc/profile sets HISTCONTROL=ignoredups unless the variable already says ignorespace, so the same session there keeps the line with the leading space:

To get the Ubuntu behaviour on RHEL, add HISTCONTROL=ignoreboth to your ~/.bashrc. Either way, the leading space is a convenience, not a secret store: a password typed as an argument (mysql -pSecret, curl -u user:pass) is visible to other users while the command runs, and the next lesson shows better places for secrets. The history file is also one of the first places an investigator, or an intruder, reads on a compromised account.

Aliases and ~/.bashrc

An alias is a short name that bash replaces with a longer command when it is the first word of a line you type. Ubuntu's default ~/.bashrc defines a few:

ll is ls -alF, and ls itself is an alias that adds colour. Aliases belong to interactive shells: ~/.bashrc defines them when a shell starts at a prompt, and bash expands them only there. The terminals in this course run each command in a non-interactive shell, as a script does, and in one of those ll does not exist. bash -i starts an interactive shell (with no terminal attached it prints two warnings about job control, which do not matter here), and type then reports the alias:

To add your own, append it to ~/.bashrc:

alias ports='ss -tln'

A new interactive shell knows ports; to use it in the shell you already have open, run source ~/.bashrc, which runs the file in your current shell. A script, a cron job or a command given to ssh does not, and fails with status 127, so scripts should spell out the real command. \ls or command ls bypasses an alias for one command. An alias such as rm='rm -i' feels safe, but it trains a habit that fails on every machine that does not have it. RHEL defines exactly these aliases for root, and Ubuntu does not:

An administrator used to RHEL's root prompt expects rm to ask first, and on Ubuntu it does not. Even on RHEL the alias applies only in root's interactive shell, not to sudo rm or a script. When you are done with the practice alias, remove it from ~/.bashrc with sed -i "/^alias ports=/d" ~/.bashrc.

Try this

In an empty practice directory, run touch "monthly report.pdf" invoice-{01..03}.pdf .draft.pdf. Predict what echo *.pdf prints, then check: the three invoices and the monthly report, but not .draft.pdf. Set f="monthly report.pdf" and compare ls $f with ls "$f": the first fails twice, once per word. Finally run echo *.docx and explain why it prints the pattern itself, and what rm *.docx would therefore do.

Takeaway

Quote every variable and every name that could contain a space, and use single quotes for patterns meant for another program. Before any command that deletes or overwrites with a wildcard, run it once with echo in front and read what the shell will really pass.

Related