Functions, local and logging

Functions with arguments, return vs exit, local scope and a stderr logger.

Once a script does more than a handful of things, the same steps turn up in several places and the file becomes hard to follow. Functions fix that: each step gets a name, and the main part of the script reads like a list of those names. In this lesson you will define functions and pass them arguments, tell a function's two kinds of answer apart (its exit status and its output) and keep them from mixing, see what return and exit each end, keep a function's variables to itself with local, and build a script with a log helper that writes UTC timestamps to standard error and guard functions that refuse bad input before anything happens. The examples run in ~/bash-ops/func.

Define a function, pass it arguments

A function is a named group of commands. You define it with name() { commands; } and call it by name, exactly like a command, with its arguments after the name. The name() form is the POSIX one; Bash also accepts function name { ... }, but this course uses the portable form.

Inside the function, $1 is the function's first argument and $# how many it got, just as in a script. type confirms that greet is now a function and prints the definition as Bash stored it. While a function runs, its arguments hide the script's own, and the script's come back when it returns:

To use a script argument inside a function, pass it in explicitly: check_host "$1", or process "$@" for all of them. Bash reads a script from top to bottom and only knows a function once its definition has run:

The first call failed as an unknown command (status 127), because the definition on the next line had not run yet. This is why scripts define their functions first and put the code that uses them at the end.

Two answers: status and output

A function can answer in two ways. Its exit status answers a yes-or-no question; it is set by return N or, without a return, by the last command the function ran. Its output is data: whatever it prints on standard output, which the caller captures with $( ). This function answers with its status only. It accepts a job name made of letters, digits, dots, dashes and underscores that does not start with a dot or a dash:

valid_name prints nothing. The [[ =~ ]] test is its last command, so the test's status is the function's status, and if branches on it. It refused ../etc (a path that climbs out of the directory), -rf (which a command would read as options), .hidden, the empty name and a name containing a newline. This is an allowlist: it states what is allowed, and everything else is refused, including inputs nobody thought of. A denylist that blocked only / and .. would have let -rf and the newline through.

A function that produces data prints it. Functions that several scripts share often live in a file the scripts source ("Your first script" showed that source runs a file in the current shell, so the functions it defines stay available). Here is one for KEY=VALUE config files, and the file it will read:

# shellcheck shell=bash
# conf.sh: helpers for KEY=VALUE config files. Source this file; do not run it.
 
# conf_get KEY FILE: print the value of KEY. Status 1 when FILE has no such key.
conf_get() {
  local want=$1 file=$2 line
  # "|| [[ -n $line ]]" also handles a last line that has no newline at its end.
  while IFS= read -r line || [[ -n $line ]]; do
    if [[ $line == "$want="* ]]; then
      # The value is everything after the first "=", including any "=" in it (base64 ends in them).
      printf '%s\n' "${line#*=}"
      return 0
    fi
  done < "$file"
  return 1
}

# app.conf: settings for the billing service
name=billing
port=8443
log_dir=/var/log/billing
api_token=bGFiLXRva2VuLW5vdC1zZWNyZXQ=

The first line is for ShellCheck, the script checker from the lesson "Check your script": a sourced file has no shebang, so the line tells it the file is Bash. The local line makes want, file and line exist only inside the function (a later section explains why that matters). conf_get reads the file with the loop from "Loops and reading input safely", which also keeps a last line that has no newline. [[ $line == "$want="* ]] is true for a line that starts with the key and an =: the quoted part is literal text and the * matches the rest. ${line#*=} is the line with everything up to and including the first = removed, which leaves the value (the lesson "Arrays and parameter expansion" covers this kind of trimming). On a match the function prints the value and returns 0; if the loop ends without a match, it returns 1.

$? right after an assignment from $( ) is the status of the command inside it, so the caller gets both answers from one line: the value in the variable, and "found or not" in the status. The token came back whole, with the = at its end: base64, a common encoding for tokens and keys, pads its output with one or two =. user is not in the file, so conf_get returned 1 and user is empty. Without the status, a missing key and an empty value would look the same.

Many scripts split such lines with read instead, as IFS='=' read -r key value: read splits the line at = and puts the rest of it into the last name. That shorter version corrupts exactly this kind of value:

With = in IFS, read treats a single = at the very end of the line as the separator that closes the last field and removes it, so the token lost its last character, with no error and status 0. Two = at the end survive, which makes the bug look random. Taking the value with ${line#*=} keeps every character. The loop header matters as well. Add a last line without a newline, the way many programs write config files:

od -c shows the file now ends in 3 with no \n after it, and conf_get still found the key. A loop without || [[ -n $line ]] would have returned 1 for retries.

Because $( ) captures everything written to standard output, a progress message printed there becomes part of the data:

port now holds two lines, and anything that uses it as a port number fails. Send messages meant for people to standard error, which $( ) does not capture:

The message still reached the terminal, and port holds only 8443. That is the rule for every function and script in this course: data on standard output, messages on standard error, and success or failure in the status.

return ends the function, exit ends the script

return leaves the function and hands its status to the caller, which carries on. exit ends the whole shell it runs in, and when a function is called from a script, that is the script:

#!/usr/bin/env bash
# return-exit.sh: return ends a function; exit ends the whole script.
 
check_return() {
  echo "check_return: failing"
  return 1
}
 
check_exit() {
  echo "check_exit: failing"
  exit 1
}
 
check_return
echo "after check_return: status $?"
captured=$(check_exit)
echo "after \$(check_exit): status $?, captured [$captured]"
check_exit
# shellcheck disable=SC2317  # unreachable on purpose: the demo shows that it never runs
echo "after check_exit: this line never runs"

The # shellcheck disable=SC2317 line tells ShellCheck that the last line is unreachable on purpose, and the text after it says why for the next person who reads the script.

After check_return the script carried on, with status 1 in $?. $( ) runs its commands in a subshell, a copy of the shell, so the exit inside $(check_exit) ended only that copy: the script got status 1 and the captured text, and kept going. The last call ran exit in the script itself, which ended at once with status 1, and the final echo never ran. Use return in functions that report a problem to their caller, and keep exit for the few whose job is to stop the script, such as die below. Remember that a die inside $( ) stops only the subshell, so check the status of the assignment, as make-workdir.sh does.

local: keep a function's variables to itself

Variables are global by default: an assignment inside a function changes the script's variable of the same name. local name=value creates a variable that exists only while the function runs. It has a second property that surprises people who know other languages: a function called from inside that function sees the local variable too. This is called dynamic scope.

#!/usr/bin/env bash
# scope.sh: which variables a function can see and change.
target=production.db
 
show_target() {
  printf '  show_target sees target=%s\n' "$target"
}
 
set_plain() {
  target=scratch.db
}
 
set_local() {
  local target=scratch.db
  show_target
}
 
set_local
printf 'after set_local: target=%s\n' "$target"
set_plain
printf 'after set_plain: target=%s\n' "$target"

show_target has no variable of its own, and it was called from set_local, so it saw set_local's local target=scratch.db. When set_local returned, its local disappeared and the script's target was still production.db. set_plain had no local, so it overwrote the script's value, and that change stayed. In a longer script this is how a helper that reuses a common name such as file or dir breaks its caller far from the line that caused it. Declare every variable a function sets as local. One caution: write local out and out=$(cmd) on separate lines, because local out=$(cmd) hides the command's failure; the lesson "Strict mode, honestly" shows why.

A log helper and guard functions

Here are the pieces together in a script that ops work needs constantly: it creates a work directory for a named job and tells the caller where it is. The path goes to standard output so another script can capture it, progress goes to standard error, and a bad name is refused before anything touches the file system.

#!/usr/bin/env bash
# make-workdir.sh: create jobs/NAME for a job and print its full path.
# Exit status: 0 created (or already there), 1 could not create it, 2 bad or missing NAME.
 
# log LEVEL MESSAGE...: one line on standard error, stamped with the UTC time (ISO 8601).
log() {
  local level=$1
  shift
  printf '%s %s %s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$level" "$*" >&2
}
 
# die STATUS MESSAGE...: log the message as an error and end the script with STATUS.
die() {
  local status=$1
  shift
  log ERROR "$*"
  exit "$status"
}
 
# valid_name NAME: letters, digits, dot, dash, underscore; must not start with a dot or a dash.
valid_name() {
  [[ $1 =~ ^[A-Za-z0-9_][A-Za-z0-9._-]*$ ]]
}
 
make_workdir() {
  local dir="jobs/$1"
  mkdir -p -- "$dir" || return 1
  printf '%s\n' "$PWD/$dir"
}
 
(( $# == 1 )) || die 2 "usage: make-workdir.sh NAME"
valid_name "$1" || die 2 "refusing job name: '$1'"
log INFO "creating the work directory for $1"
path=$(make_workdir "$1") || die 1 "could not create jobs/$1"
log INFO "ready: $path"
printf '%s\n' "$path"

log takes a level and a message. shift removes the first argument and moves the rest down one place, so once the level is saved in a local variable, "$*" is the message, joined with spaces. date -u +%Y-%m-%dT%H:%M:%SZ prints the time in UTC in ISO 8601 form: lines from servers in different time zones sort and compare correctly, and the Z says which zone it is. die logs an error and exits with the status it is given, which makes each guard at the bottom one readable line: first the argument count, then the name. make_workdir reports its failure with return 1 and leaves the decision to the caller. mkdir -p creates missing parent directories and succeeds if the directory already exists, so running the script twice is harmless.

The two log lines went to standard error and the path to standard output. At a terminal they look alike, so capture the output to see the difference:

With 2>/dev/null the log lines are gone and path holds exactly one line, the path. A scheduled job would append standard error to a log file with 2>> make-workdir.log instead of discarding it. The guards refuse bad input with status 2, and a failure to do the job gives status 1:

Nothing was created for ../etc or -rf. When jobs was read-only, mkdir failed, make_workdir returned 1, and die 1 ran. Look at the mkdir message: the uutils mkdir that Ubuntu 26.04 uses by default does not say which path it could not create (GNU mkdir does), so the script's own error line, which names jobs/monthly, is the one that tells you what failed. Write your error messages so that they make sense without the tool's.

Try this

In a new directory with a copy of make-workdir.sh, write remove-workdir.sh NAME, which deletes jobs/NAME. Copy log, die and valid_name into it and keep the same two guards. Add a third guard that stops with die 1 "no such job: $1" unless jobs/$1 is a directory, then delete with rm -r -- "jobs/$1" || die 1 "could not remove jobs/$1" and log removed jobs/$1. Create a job with ./make-workdir.sh nightly 2>/dev/null, then check:

1. .., -rf and '' (the empty name), each followed by echo "status: $?": expect ERROR refusing job name: '..' (and so on) and status 2 every time. Without the name guard, a name such as ../reports would point rm -r at a directory outside jobs.

2. ./remove-workdir.sh weekly: expect ERROR no such job: weekly and status 1.

3. ./remove-workdir.sh nightly: expect INFO removed jobs/nightly, status 0, and ls -A jobs printing nothing.

Takeaway

Give each function one job and one way to answer: a status for yes or no, or data on standard output with a status that says whether it worked, and messages on standard error. Make its variables local, use return inside it, and keep exit for a die that the script's guards call before any work starts.

Related