A real ops script, end to end

Everything applied: a safe backup-and-report script.

Every earlier lesson in this course handed you one tool at a time: a variable here, a loop there, a test in the middle. This lesson bolts them together on one machine. You are going to build a backup-and-report script for operations work (ops, the day-to-day running and babysitting of servers). Point it at a folder and it packs that folder into a single compressed archive, files that archive away under a timestamped name, and prints a short report of what it did. It works like the night-shift clerk who copies the day's paperwork into a labeled box, seals the box, and leaves a note on your desk saying how many pages went in.

What the script has to do

Before you write a line, say the job out loud in plain steps. The script takes one argument, the folder to back up, and refuses to run without it. It checks that the folder is really there. It makes itself a private scratch area to work in, the way a chef clears a corner of the counter before cooking. It walks through the files, sorting them into piles by size and skipping the noisy log files. It squeezes the whole folder into one .tar.gz file (a tarball, the Unix world's answer to a zip file). It takes a fingerprint of that file so you can prove later that nobody altered it. Then it prints a report and cleans up after itself. Every one of those steps is a lesson you have already met, now wired in sequence.

The safety rail: strict mode and a cleanup trap

The first real line is the one that stops a small mistake from turning into a disaster. `set -euo pipefail` is the circuit breaker for your script, the switch that cuts the power the instant something shorts out. It bundles three settings. `-e` (short for errexit, meaning "exit on error") halts the script the moment any command fails, instead of plowing ahead with half the job done. `-u` (nounset, "no unset variables") turns a mistyped variable name into a loud error instead of a silent empty string. That one matters: it means a fat-fingered `rm -rf "$dr/"` on a misspelled `$dir` blows up safely rather than quietly becoming `rm -rf /` and erasing the whole disk. `-o pipefail` makes a chain of piped commands report failure if any link in the chain failed, not only the last one. Leave these three off and a backup that half-worked can still exit with the code for success and swear to you that all is well.

The second guardrail is the `trap`. A trap is a standing order you leave with the shell: whenever this script exits, for any reason at all, run this cleanup step first. Think of it as telling the last person out of the office to switch off the lights, no matter which door they leave by. Ours deletes the scratch directory. A clean finish, a crash under strict mode, someone jabbing Ctrl-C to kill it: the temporary files get swept up every time. You register the trap once with `trap cleanup EXIT` and then forget about it. Notice the `work=""` line above the function. The variable has to exist before the trap could ever fire, so the cleanup never trips over a name that was never set.

#!/usr/bin/env bash
set -euo pipefail
 
# Where finished archives land. Override with the BACKUP_DIR env var.
readonly BACKUP_DIR="${BACKUP_DIR:-$HOME/backups}"
 
# A timestamped logger so every line says when it happened.
log() {
    printf '%s  %s\n' "$(date +'%Y-%m-%d %H:%M:%S')" "$*"
}
 
# The cleanup crew. Runs on ANY exit: success, error, or Ctrl-C.
work=""
cleanup() {
    if [[ -n "$work" && -d "$work" ]]; then
        rm -rf -- "$work"
        log "cleaned up scratch dir $work"
    fi
}
trap cleanup EXIT

Strict mode has sharp edges, so learn them before they cut you. Some commands fail on purpose as part of normal use. `grep`, the text-search tool, returns a failure code when it finds no match, and under `-e` that empty result kills your whole script. When a command is allowed to come up empty, say so out loud: write `grep pattern file || true`, or wrap the command in an `if`. And because `-u` insists every variable exist before you read it, you declare the array piles up front rather than on first use. Strict mode does not check your logic. It only stops you from ignoring failures you never planned for.

Trust nothing: validating the input

A script that runs as an unattended ops job, sometimes as a powerful user with the keys to the whole machine, should be suspicious of whatever it is handed, the way a bouncer checks every ID at the door. Two checks do the work here. First, count the arguments. Exactly one, or the script prints a usage line to standard error (the separate output channel meant for diagnostics, written with `>&2`) and quits with code 2. Second, confirm the argument really is a directory using the `-d` test; if it is not, log an error and quit with code 1. The different numbers matter because whatever launched this script reads that number to decide what to do next. That caller might be cron (the scheduler that fires jobs on a timer), a continuous integration pipeline (CI, the automated system that builds and tests your code), or another script. Zero means success. Anything else means stop and look.

# 1. Check the input before doing anything.
if [[ $# -ne 1 ]]; then
    echo "usage: $0 <source-directory>" >&2
    exit 2
fi
src="$1"
 
if [[ ! -d "$src" ]]; then
    log "ERROR: '$src' is not a directory"
    exit 1
fi

Two wrong ways to call it, two distinct exit codes. An automated caller can now tell "you used it wrong" (code 2) apart from "the target is missing" (code 1) and react to each differently, without ever reading a word of the human-facing message.

Walking the files: loop, array, branch

Now the real work. You want to visit every file under the source folder. The safe way to list them is `find ... -print0`, which ends each name with an invisible zero byte instead of a newline, like dropping an unmarkable fence between one name and the next. Why go to the trouble? Filenames are allowed to contain spaces, newlines, and even names like `--checkpoint-action` that a command might read as an instruction to itself rather than a file. Pairing `-print0` with `read -r -d ''` (read one record, and treat that zero byte as the end of it) means a file called `my report.txt` arrives as a single item, not two. For the same reason, every variable gets wrapped in double quotes: you write `"$file"`, never a bare `$file`.

Inside the loop, three arrays act as sorting bins: `big`, `small`, and `skipped`. An array is a numbered list that grows as you add to it, like a row of pigeonholes you keep filling, and `${#big[@]}` reads back how many things are sitting in one of them. A `case` statement checks each filename. Anything ending in `.log` drops straight into the skip bin, because raw logs are noisy and often hold passwords or tokens you would rather not copy around. Everything else runs through an `if` that reads the file's size in bytes with `stat` and files it under big or small. Branching and counting, the same moves the earlier lessons drilled, now working over real files.

work="$(mktemp -d)"
log "working in $work"
mkdir -p -- "$BACKUP_DIR"
 
declare -a big=()
declare -a small=()
declare -a skipped=()
count=0
 
while IFS= read -r -d '' file; do
    count=$((count + 1))
    bytes="$(stat -c '%s' -- "$file")"
    case "$file" in
        *.log)
            skipped+=("$file")
            log "skipping log file $file"
            ;;
        *)
            if (( bytes >= 1024 )); then
                big+=("$file")
            else
                small+=("$file")
            fi
            ;;
    esac
done < <(find "$src" -type f -print0 | sort -z)

Make the archive, then prove it

With the files counted, `tar -czf` packs the source folder into one file squeezed down with gzip (a standard Unix compression format), the way a vacuum bag crushes a thick duvet to a fraction of its size: `c` for create, `z` for gzip, `f` for the output filename that comes next. Adding `-C "$(dirname ...)"` and then the bare folder name keeps the paths inside the archive tidy, so you get `demo/config.yml` and not `/tmp/tmp.abc/demo/config.yml`. The whole `tar` call sits inside an `if` so you can see whether it worked. On success you log the path. On failure you catch the exit code in `rc`, report it, and stop. Then `sha256sum` takes a fingerprint of the archive using SHA-256 (Secure Hash Algorithm, 256-bit), a 64-character string of hexadecimal digits (hex, base-16: the digits 0 through 9 plus a through f) that changes completely if even one byte of the file changes. It works like a tamper-evident sticker sealed across the box flap. Save that fingerprint and months later you can prove the backup you are about to restore is the one you actually made, not a swapped-in copy.

stamp="$(date +'%Y%m%d-%H%M%S')"
base="$(basename -- "$src")"
archive="$BACKUP_DIR/${base}-${stamp}.tar.gz"
 
if tar -czf "$archive" -C "$(dirname -- "$src")" "$base"; then
    log "archive written to $archive"
else
    rc=$?
    log "ERROR: tar failed (exit $rc)"
    exit 1
fi
 
sum="$(sha256sum -- "$archive" | cut -d ' ' -f1)"

The whole thing

Here it is assembled. Read it top to bottom and the shape of the job shows through: the safety rail, the input checks, the private scratch space, the sorting loop, the archive with its exit-code check, and the report at the end.

#!/usr/bin/env bash
#
# safe-backup.sh -- archive a directory and print a report.
# Usage: ./safe-backup.sh <source-directory>
 
set -euo pipefail
 
# Where finished archives land. Override with the BACKUP_DIR env var.
readonly BACKUP_DIR="${BACKUP_DIR:-$HOME/backups}"
 
# A timestamped logger so every line says when it happened.
log() {
    printf '%s  %s\n' "$(date +'%Y-%m-%d %H:%M:%S')" "$*"
}
 
# The cleanup crew. Runs on ANY exit: success, error, or Ctrl-C.
work=""
cleanup() {
    if [[ -n "$work" && -d "$work" ]]; then
        rm -rf -- "$work"
        log "cleaned up scratch dir $work"
    fi
}
trap cleanup EXIT
 
# 1. Check the input before doing anything.
if [[ $# -ne 1 ]]; then
    echo "usage: $0 <source-directory>" >&2
    exit 2
fi
src="$1"
 
if [[ ! -d "$src" ]]; then
    log "ERROR: '$src' is not a directory"
    exit 1
fi
 
# 2. A private scratch space that only we can read (mktemp gives 0700).
work="$(mktemp -d)"
log "working in $work"
mkdir -p -- "$BACKUP_DIR"
 
# 3. Walk every file and sort it into a bucket.
declare -a big=()
declare -a small=()
declare -a skipped=()
count=0
 
while IFS= read -r -d '' file; do
    count=$((count + 1))
    bytes="$(stat -c '%s' -- "$file")"
    case "$file" in
        *.log)
            skipped+=("$file")
            log "skipping log file $file"
            ;;
        *)
            if (( bytes >= 1024 )); then
                big+=("$file")
            else
                small+=("$file")
            fi
            ;;
    esac
done < <(find "$src" -type f -print0 | sort -z)
 
# 4. Build the archive, and check that tar actually succeeded.
stamp="$(date +'%Y%m%d-%H%M%S')"
base="$(basename -- "$src")"
archive="$BACKUP_DIR/${base}-${stamp}.tar.gz"
 
if tar -czf "$archive" -C "$(dirname -- "$src")" "$base"; then
    log "archive written to $archive"
else
    rc=$?
    log "ERROR: tar failed (exit $rc)"
    exit 1
fi
 
sum="$(sha256sum -- "$archive" | cut -d ' ' -f1)"
 
# 5. The report.
printf '\n===== backup report =====\n'
printf 'source      : %s\n' "$src"
printf 'files seen  : %d\n' "$count"
printf 'big (>=1KB) : %d\n' "${#big[@]}"
printf 'small       : %d\n' "${#small[@]}"
printf 'logs skipped: %d\n' "${#skipped[@]}"
printf 'archive     : %s\n' "$archive"
printf 'sha256      : %s\n' "$sum"
printf '=========================\n'

A real run

A sample `demo/` folder holds four things: a small configuration file, a one-line note, a 4 KB (kilobyte, about four thousand bytes) binary file (raw non-text data, the kind of blob a program or an image is made of), and a service log. Run the script against it and send the finished archive to a local `backups` folder. Watch the order in which things print.

Read the order of events. The report prints, and then the very last line is the cleanup running. That is the EXIT trap firing on the way out: the scratch directory lived only for the length of the run and was gone the moment the script finished. Four files seen, one log skipped, one file at or above a kilobyte, two smaller ones, and a fingerprint you can check against the archive on disk any time with `sha256sum -c`. Every number in that report came from an array count or an exit-code test you wrote by hand.

To grow the script, add a retention step that deletes archives older than 30 days, or push the report to your alerting channel on the days `skipped` comes back higher than you expect. The skeleton holds up under either change, because the habits underneath it are the same ones that make any ops script safe to leave running alone: check the input before you trust it, quote every variable, confirm what your tools return, and clean up on the way out.

Related