Arrays and parameter expansion

Argument arrays, associative arrays and ${var:?} guards on destructive paths.

A variable holds one string, but scripts constantly need lists (hosts, files, the arguments of a command) and small edits to strings (a file name without its directory, a default when a setting is missing). This lesson covers indexed arrays, the safe way to build a command's arguments, parameter expansions for defaults, required values, trimming and replacing, and associative arrays for counting by key. It also gives you the rule that stops a destructive command from running with an empty path. Arrays are Bash-only: Dash rejects them, as "Your first script" showed. The examples run in ~/bash-ops/arrays.

Indexed arrays: a list that keeps its items whole

name=(one two three) creates an indexed array, a list whose elements are numbered from 0. Each word inside the parentheses is one element, so quote an element that contains a space:

${#hosts[@]} is the number of elements, ${hosts[0]} the first and ${hosts[-1]} the last. "${hosts[@]}" expands to every element as a separate word, the array version of "$@", which is why db 01 stayed whole. The other two forms behave like $* and "$*":

Unquoted, the elements were split again; "${hosts[*]}" joined them into one word. Use "${hosts[@]}" whenever you pass the elements on, and [*] only inside a message. +=( ) appends, and unset removes an element:

After the unset, the indexes are 0, 2 and 3. Bash does not renumber, so loop over the elements with "${hosts[@]}" instead of counting from 0 to the length. The quotes in unset 'hosts[1]' matter as well: unquoted, hosts[1] is a glob pattern, and if the directory holds a file named hosts1, Bash replaces the pattern with that name and unsets a variable called hosts1 instead (the lab checked this).

Build a command's arguments in an array

The most useful job for an array is holding the arguments of a command that a script puts together as it goes. The examples search a small SSH log. It is not the auth.log from "Exit status and tests", so save this one as auth.log in ~/bash-ops/arrays. Its last line is crafted on purpose; the section on associative arrays explains it:

2026-09-27T10:01:12.100211+00:00 web01 sshd-session[3101]: Failed password for invalid user admin from 203.0.113.7 port 50214 ssh2
2026-09-27T10:01:15.441873+00:00 web01 sshd-session[3101]: Failed password for invalid user admin from 203.0.113.7 port 50214 ssh2
2026-09-27T10:02:40.918342+00:00 web01 sshd-session[3120]: Accepted publickey for deploy from 198.51.100.24 port 41822 ssh2
2026-09-27T10:03:05.200117+00:00 web01 sshd-session[3134]: Failed password for root from 192.0.2.44 port 38810 ssh2
2026-09-27T10:04:51.662045+00:00 web01 sshd-session[3150]: Failed password for root from 203.0.113.7 port 50388 ssh2
2026-09-27T10:05:09.003981+00:00 web01 sshd-session[3161]: Failed password for invalid user x from 198.51.100.99 port 1 from 203.0.113.7 port 50412 ssh2

The tempting alternative to an array is a string:

opts was split at every space, and the single quotes inside it were ordinary characters, because the results of an expansion are never parsed again. grep received -n, -e, 'Failed and password': it searched for the text 'Failed in two files, one of them named password', which does not exist. An array keeps each argument exactly as written:

declare -p prints a variable the way Bash stores it, the quickest way to check an array while you write a script. Here is a script whose arguments depend on its input:

#!/usr/bin/env bash
# search-auth.sh: show failed SSH password logins in a log, optionally for one user only.
# Usage: search-auth.sh LOGFILE [USER]
# Exit status (grep's): 0 lines found, 1 none found, 2 usage error or unreadable file.
if [[ $# -lt 1 || $# -gt 2 ]]; then
  printf 'usage: search-auth.sh LOGFILE [USER]\n' >&2
  exit 2
fi
log=$1
user=${2:-}
# --fixed-strings: the user name is data, never a regular expression.
args=(--line-number --fixed-strings)
if [[ -n $user ]]; then
  args+=(--regexp "Failed password for $user from")
else
  args+=(--regexp 'Failed password for')
fi
grep "${args[@]}" -- "$log"

args starts with two options, and each += adds one pattern argument, spaces and all. --fixed-strings (-F) makes grep treat the pattern as plain text: the user name comes from outside the script, and without -F a name such as .* would be a regular expression matching every user. -- before $log keeps a file name that starts with a dash from being read as an option. The script ends with grep, so grep's status (0 found, 1 none, 2 error) is the script's, as its header says.

Only root's two lines matched, and .* was searched for literally and found nothing (status 1). An array element can never turn into two arguments. It can still be read as an option if it starts with a dash, unless it comes after -- or is the value of an option, as the user name is here with --regexp.

Defaults, required values and the destructive-path rule

Parameter expansion can react to a variable that is unset or empty. There are four forms:

${region:-eu-west-1} used the default because region is unset, and ${empty:-fallback} because empty is empty. Without the colon, ${empty-fallback} replaces only an unset variable, so the empty value came through. := also assigns the default. : is a command that does nothing, but its arguments are still expanded, which makes : "${LOG_DIR:=...}" the usual way to set a default at the top of a script. ${v:+word} gives word only when the variable has a value, which suits an optional flag:

Unquoted, the empty case produced no argument at all. Quoted, it produced an empty argument, which a command will misread as an empty file name or reject. An if that adds to an argument array, as in search-auth.sh, says the same thing more clearly.

Defaults suit settings. They are the wrong tool for a value that decides what gets deleted. Here is a cache with two environments, and a copy to experiment on:

A cleanup that empties one environment's cache might run rm -rf -- cache/"$TARGET"/*. If TARGET is empty (a typo in a cron file, a variable missing from a CI job), the path becomes cache//*:

Both environments are gone, and rm -rf said nothing. A default such as ${TARGET:-staging} would not help: a forgotten variable would quietly empty staging instead, which is still an action nobody asked for. For a path that decides what gets deleted, demand the value:

#!/usr/bin/env bash
# purge-cache.sh: empty the cache directory of one environment.
# Usage: TARGET=staging|prod ./purge-cache.sh
cache_root=cache
rm -rf -- "${cache_root:?}/${TARGET:?TARGET must be set to staging or prod}/"*
echo "emptied $cache_root/$TARGET"

${TARGET:?message} expands to the value when there is one; otherwise the shell prints the message and, in a script, exits before the command runs. ${cache_root:?} does the same with a standard message for a variable the script sets itself, so a later edit that breaks it cannot turn the path into /*. ShellCheck reports rm -rf "$dir/"* without it as SC2115.

With TARGET unset, the script stopped at line 5 with status 1, and both prod files are still there. set -u, from the lesson "Strict mode, honestly", would not have caught an empty value: it stops only on variables that were never set. :? treats empty and unset alike.

Trim and replace

Four operators remove a pattern from one end of a value: # the shortest match at the start, ## the longest match at the start, % the shortest at the end and %% the longest at the end. The patterns are glob patterns, as in case:

${path##*/} removed everything up to the last /, leaving the file name, and ${path%/*} removed the last / and what follows it, leaving the directory. On the file name, %%.* kept the part before the first dot, #*. removed up to the first dot, %.gz removed one suffix, and ${#file} is the length. All of this happens inside Bash, without starting basename, dirname or sed for each name, which adds up in a loop over thousands of files. A value with no / at all comes back unchanged from ${path%/*}, so check for that when the input might be a bare file name.

${var/pattern/string} replaces the first match and ${var//pattern/string} every match (both Bash-only):

Every space in the file name became _. For the connection string, the aim is to hide the password before the line is logged. pass=* matched to the end of the string, because * matches anything, ; included, so the port disappeared too. shopt -s extglob turns on Bash's extended patterns, in which *([^;]) means "any number of characters other than ;", and only the password was replaced (shopt -u extglob turns them off again). Treat redaction as a last line of defence: the better fix is a secret that never enters the string, as "Variables, quotes and command output" explained.

Associative arrays: counting by key

An associative array uses strings as keys instead of numbers, which makes it the tool for counting or looking things up by name. It must be declared with declare -A. This script counts failed SSH password attempts per source address in a small log:

#!/usr/bin/env bash
# fail-count.sh: count failed SSH password attempts per source address.
# Usage: fail-count.sh LOGFILE
if [[ $# -ne 1 || ! -r $1 ]]; then
  printf 'usage: fail-count.sh LOGFILE (a readable file)\n' >&2
  exit 2
fi
declare -A fails=()
while IFS= read -r line || [[ -n $line ]]; do
  [[ $line == *'Failed password for '*' from '* ]] || continue
  addr=${line##* from }   # longest match: everything up to the LAST " from "
  addr=${addr%% *}        # keep the first word: the address
  fails[$addr]=$(( ${fails[$addr]:-0} + 1 ))
done < "$1"
for addr in "${!fails[@]}"; do
  printf '%5d  %s\n' "${fails[$addr]}" "$addr"
done | sort -rn

The [[ ]] line skips anything that is not a failed password; the quoted parts are literal text and the * between them are wildcards. ${line##* from } removes everything up to the last occurrence of the word from (with a space on each side), and ${addr%% *} keeps the first word, the address. ${fails[$addr]:-0} is the current count, 0 the first time an address appears. "${!fails[@]}" expands to the keys, in no defined order, so the output goes through sort -rn (numeric, largest first).

203.0.113.7 failed four times and 192.0.2.44 once. The last line of auth.log shows why the script uses ##. The user name in a failed-login line is whatever the client sent, so an attacker controls that part of the line, and this one was crafted with the user name x from 198.51.100.99 port 1:

# stopped at the first from and blamed an address that never connected. A job that blocked addresses from these counts would have blocked an innocent host, chosen by the attacker. ## takes the last from, the one sshd writes after the user name. When part of a line comes from someone else, anchor your parsing on the parts you control. Leaving out declare -A produces no error at all:

Without the declaration, counts is an indexed array, and an index is evaluated as arithmetic: web, db and cache were read as the names of unset variables, which count as 0. All three assignments wrote element 0, and only the last value survived. That arithmetic also means a key taken from input could run code, as "Exit status and tests" showed for [[ $x -eq 0 ]]. Declare every associative array before its first use. Inside a function, declare makes the variable local, like local.

Try this

${TARGET:?} stops an empty value, not a wrong one. In a throwaway directory, give the lesson's purge-cache.sh a cache/prod directory with one file and a file app.conf next to cache, and think about what TARGET=.. would do. The lab ran exactly that: the script printed emptied cache/.. with status 0, and app.conf and the whole cache directory were deleted, because cache/.. is the directory that holds cache.

Fix it with an allowlist in front of the rm: case ${TARGET:-} in, a staging | prod) ;; branch, and a default branch that prints purge-cache.sh: TARGET must be staging or prod, got [VALUE] to standard error and exits 2. Then check:

1. TARGET=.. ./purge-cache.sh prints got [..] and exits 2, and nothing is deleted.

2. ./purge-cache.sh with TARGET unset prints got [] and exits 2.

3. TARGET=prod ./purge-cache.sh prints emptied cache/prod and exits 0, and find . -type f no longer lists the file in cache/prod.

Takeaway

Keep lists, and above all a command's arguments, in arrays and expand them as "${name[@]}", and declare associative arrays with declare -A. For any path that decides what gets deleted, use ${var:?} so an empty value stops the script, plus an allowlist so a wrong value cannot get through either.

Related