Test yourself

Advanced scripting for DevSecOps

Final exam · 47 questions · answers explained as you pick

Shell in production

18 questions

01A gate script starts with set -euo pipefail and shopt -s lastpipe, then counts with grep -F 'Failed password' -- "$log" | while IFS= read -r _; do failed=$((failed + 1)); done before it prints a verdict. The log it is given has mode 000. What happens?

AIt prints failed logins: 0 and PASS with exit 0, because the loop now runs in the current shell and grep's error goes unseen.

Incorrect — That is what the process substitution version did. With lastpipe, grep is still a stage of a pipeline, so pipefail and errexit see its status.

Bgrep reports Permission denied, and the script stops at the pipeline with status 2 before it prints a verdict.

Correct — Under pipefail the pipeline's status is grep's 2, errexit ends the script on that line, and no PASS can be printed: the gate fails closed.

CThe count stays 0 and the gate passes, because lastpipe takes effect when job control is on, and scripts run with it off.

Incorrect — It is the other way round: lastpipe applies when job control is off, the default in scripts, so the loop does run in the current shell.

Dgrep prints its error and the gate still prints PASS: pipefail records the failure, but errexit ignores a failed first stage.

Incorrect — errexit acts on the status of the whole pipeline, and pipefail makes that status grep's 2, so the script stops at that line.

02A drift check runs diff <(grep -v '^#' -- expected.conf) <(grep -v '^#' -- live.conf), then wait "$!", and treats a status above 1 as an error. One night it reported "no drift" although expected.conf had been deleted and live.conf held only comments. Why did the check pass?

Adiff exits 0 when one of its inputs cannot be opened, so the check should also test diff's own status for 2.

Incorrect — diff opened both /dev/fd paths without trouble and read two empty streams; it never saw the missing file, so its 0 described what it read.

BA bare wait would have caught it, because without arguments it returns the worst status of the background jobs it waited for.

Incorrect — A wait without arguments waits for every child and returns 0; it reports none of their statuses.

Cgrep exits 1 for a missing file, the same code as "no lines selected", so no wait could tell the two apart.

Incorrect — grep exits 2 for an error such as a missing file; 1 means that no lines were selected, which is why the check accepts it.

D$! named the last substitution, so wait saw live.conf's grep (1); the exit 2 for expected.conf went uncollected.

Correct — Open each producer with exec {fd}< <(...), keep one PID per producer and wait for each, as the lesson did for two greps.

03An operator tries the stop logic by hand at an interactive prompt: setsid ./check-host.sh demo 30 0 &, then kill -TERM -- "-$!". the kill fails and the check keeps running, yet the same two lines work inside sweep.sh. What explains the difference?

AAt a prompt setsid forks, so $! is a parent that already exited; use setsid -f and let the child record its PID.

Correct — In a script setsid need not fork and $! is the new group's leader; at a prompt & already makes a group, setsid forks, and the real group ID is a PID you never saw.

BInteractive shells make background jobs ignore SIGTERM, so the signal reached the group and each member dropped it.

Incorrect — Background jobs of an interactive shell do not ignore SIGTERM; kill failed because no process group had that ID.

CA negative PID is read as an option at a prompt, so kill needs -s TERM instead of -TERM in front of it there.

Incorrect — The -- already stops kill from reading -PID as an option, in scripts and at a prompt alike.

Dsetsid does nothing when a terminal is attached, so the check stayed in the prompt's group and -$! pointed nowhere.

Incorrect — setsid still creates a new session; the problem is which PID $! holds, not whether the session was made.

04shellcheck -o check-set-e-suppressed,check-extra-masked-returns refresh-naive.sh reports SC2312 on the printf '%s\n' "$(fetch)" line and SC2310 on if ! install_list; then. The team wants install_list to stay correct however a later caller invokes it. Which change does that?

ASilence SC2310 with a directive, because if ! is the intended form and the script header already turns on strict mode.

Incorrect — Called in a condition, the function runs with errexit off whatever the header says; the directive hides the bug.

BAdd set -E, which carries errexit into functions that are called as the condition of an if or a !.

Incorrect — set -E makes the ERR trap inherited; it does not bring errexit back inside a tested function.

CMake each step check itself with || and return (download, write, mv), so the function stops relying on errexit.

Correct — Explicit checks work in any calling context, and a plain body=$(fetch) with its own check also clears SC2312.

DReplace the if with a || list that calls a die function, which takes the call out of any condition context.

Incorrect — The left side of a || list is a tested context too, so errexit stays off inside install_list.

05The fail-closed refresh.sh installs a new blocklist that passed its checks: at least one entry and no invalid line. The single entry is 0.0.0.0/0, and the firewall now blocks every source. What was missing?

APlausibility gates: the checks prove format, not sense, so add a prefix floor, a protected allowlist and a shrink limit.

Correct — Syntax checks cannot judge meaning; a floor on prefix length, ranges that may never be blocked and a limit on change against the current list can.

BNothing in the script: its pattern rejects 0.0.0.0/0, so another process must have written the list after the refresh.

Incorrect — The entry pattern accepts any dotted quad with an optional prefix length, 0.0.0.0/0 included; the refresh installed it.

CAn atomic install: mv -f copied the file in pieces, and the firewall read it while one line was present.

Incorrect — The temp file sits in the list's own directory, so mv is a rename and readers see the old or the new file whole.

DA stricter curl: without --fail-with-body a proxy error page arrives with status 200 and passes the checks.

Incorrect — An HTML page would contain lines that fail the pattern and be refused; this file was a single well-formed entry.

06The nightly blocklist refresh leaves error: tmp=$(mktemp -- "$list.XXXXXX") exited 1 on stderr, followed by at ./refresh.sh:26 in refresh_list and at ./refresh.sh:36 in main. How should the on-call engineer read it?

AA function called main failed at line 36 and then called refresh_list, whose mktemp at line 26 was a second failure.

Incorrect — The stack reads from the failure outward; main is the name Bash gives the top level, not a function in the script.

BThe trap fired twice, once per stack line, so one failure was reported from a subshell and again from the main shell.

Incorrect — Each at line is one frame of a single report; trace.sh reports from the main shell and exits.

CThe list was installed, and mktemp then failed during cleanup, so the new list is live but a temporary file was left.

Incorrect — mktemp creates the temp file before the download; it failed first, so no install happened.

Dmktemp failed at line 26 in refresh_list, called from line 36 of the top level; no list was installed.

Correct — caller frame 0 is the failing call and "main" is Bash's name for the top level; the unplanned failure stopped the run before the install, with a non-zero status.

07A timer starts a Type=oneshot unit whose script cleans up on SIGTERM and then re-raises it, a pattern copied from a long-running service that stops cleanly. Stopping the oneshot unit mid-run leaves code=killed, status=15/TERM and result 'signal' in its log, and the unit is marked failed. Why, and what fixes it?

AThe re-raise came too late, after systemd had sent SIGKILL, so TimeoutStopSec= must exceed the cleanup time.

Incorrect — The status is 15/TERM, not 9/KILL, and the result is 'signal', not 'timeout'; the cleanup finished in time.

BFor Type=oneshot a death by SIGTERM is not a clean exit; add SuccessExitStatus=TERM to the unit.

Correct — systemd.service(5) counts a SIGTERM death as clean except for oneshot units; declaring TERM (or 143 for a script that exits 143) records the stop as a success.

CA oneshot unit has to exit with 143 instead, since systemd accepts 143 as a clean stop for batch jobs.

Incorrect — Exit 143 is an ordinary failing status for every unit type unless SuccessExitStatus= names it.

DKillMode=control-group signalled the children first, so the re-raise happened in a child and was counted.

Incorrect — The main process itself died of SIGTERM, as the status line shows; which children were signalled does not change that verdict.

08A container's entrypoint is a Bash script with trap cleanup EXIT and no TERM trap; it runs ./batch-worker in the foreground. On pod deletion the container stops within a second with exit code 143, but the worker's batch is cut off midway. What happened?

AThe kernel dropped SIGTERM because PID 1 had no TERM trap, and the grace period then ended with SIGKILL.

Incorrect — That is the no-trap case, which ends after the grace period with 137. Here the EXIT trap gave Bash a handler, so the signal arrived.

BBash forwarded SIGTERM to the worker, which has no handler of its own, so the worker died first and Bash then exited 143.

Incorrect — Bash does not forward signals to its children; the worker died because PID 1 exited, not from a forwarded signal.

CThe EXIT trap gave Bash a TERM handler, so it cleaned up and exited at once, and PID 1 exiting killed the worker.

Correct — An EXIT trap makes the stop fast, not graceful: forward SIGTERM to the worker and wait for it, or run the worker under an init such as tini.

DterminationGracePeriodSeconds was too short for the batch, so raising it would let the worker finish its work.

Incorrect — The container exited within a second, long before a grace period could run out; more time changes nothing.

09A batch unit uses the default KillMode. Its script sets a flag in a TERM trap and is meant to stop at the next item boundary. Stopped one second into an item, the unit logs upload 1: start, then Terminated ./upload.sh "$item", then status=143/n/a. What does this sequence tell you?

ATimeoutStopSec ran out during the upload, and systemd killed the whole unit before the flag could be read.

Incorrect — A timeout would show "State 'stop-sigterm' timed out" and a result of 'timeout'; this stop was immediate.

BThe flag-and-check design cannot work in Bash, because a trap runs before the foreground command it interrupted returns.

Incorrect — Bash runs the trap after the foreground command returns, which is exactly what makes the item-boundary drain work.

CThe script exited 143 from its trap, which systemd records as a failure; re-raising the signal instead would fix it.

Incorrect — The trap sets a flag and nothing more; the 143 came from the killed child through errexit, before any re-raise could run.

Dcontrol-group sent SIGTERM to the upload as well; it died with 143 and set -e ended the batch. Use KillMode=mixed.

Correct — With mixed the main process alone gets SIGTERM, the upload finishes, and the script re-raises at the boundary.

10A nightly scan runs as a Type=oneshot service with RuntimeMaxSec=30min. One morning systemctl status shows it still "activating" after six hours, and nobody was paged. What is wrong?

ARuntimeMaxSec does not apply to oneshot units, which stay activating until the command exits; use TimeoutStartSec=.

Correct — For oneshot units the limit is TimeoutStartSec, which defaults to infinity; the lesson's 2-second test failed the unit with result 'timeout'.

BRuntimeMaxSec counts CPU time, and a scan that hangs waiting on the network uses almost none of its budget.

Incorrect — RuntimeMaxSec is wall-clock time; the problem is that it has no effect on this unit type.

CThe limit fired, but the scan ignores SIGTERM; add TimeoutStopSec= so that systemd follows up with SIGKILL after a while.

Incorrect — Nothing fired: RuntimeMaxSec has no effect on a oneshot unit, so the scan was never signalled.

DA timer overrides the time limits of the service it starts, so RuntimeMaxSec belongs on the timer unit.

Incorrect — Time limits belong to the service unit; the timer merely starts it.

11A root job runs write-secure.sh /srv/drop/reports/today.txt. reports is a symlink that the uploads team can repoint, and today it points at a root-owned directory with mode 0755. The script stops with write-secure: refusing /srv/drop/reports: it is a symlink; name the real directory. A teammate calls that overcautious, since the target passes the owner and mode checks. Who is right?

AThe teammate: the checks look at the directory the link points to today, so the write would land in a safe place.

Incorrect — Today's target proves nothing about the next run; whoever can repoint the link picks where a root-owned write lands.

BThe teammate: mv -T replaces a symlink instead of following it, so a link on the path cannot redirect the file.

Incorrect — -T governs the last name alone; a symlinked directory earlier in the path is still followed.

CThe script: whoever can repoint reports picks where the next write lands, and stat without -L sees the link.

Correct — Owner and mode checks can vouch for a real directory; a link on the way hands the choice to whoever controls the link.

DThe script, though because mktemp refuses to create files inside a directory that it reached through a symlink.

Incorrect — mktemp follows a symlinked directory like any path lookup; the refusal is the script's own rule.

12A job writes a report by creating $tmp with mktemp in spool/ and running mv -f -- "$tmp" spool/report. After one run, spool/report is still an old symlink to /home/app/cache, and a .tmp.XXXXXX file has appeared inside /home/app/cache. What happened?

Amktemp followed the symlink and created the temp file in the link's target directory instead of in spool/.

Incorrect — mktemp created the file in spool/ under a new random name; the link played no part until the move.

BThe destination was a link to a directory, so mv moved the file into it; mv -T makes the name the target.

Correct — Without -T mv treats a directory destination, even one reached through a link, as a place to move into; --no-target-directory replaces the name.

CThe rename crossed filesystems, so mv copied through the symlink instead of renaming within spool/.

Incorrect — The temp file and the destination are in the same directory, so this was a rename; the problem is how mv chose its target.

D-f makes mv dereference symlinks at the destination, so dropping -f would have replaced the link.

Incorrect — -f suppresses prompts; without it mv would still move the file into the directory the link names.

13A wrapper runs ssh svc@db01 "stat -- ${path@Q}". The account svc has /bin/sh (dash) as its login shell, and one path contains a newline. What does the remote side receive?

AA second command: the newline inside the quoted value ends stat, and the rest of the path runs as a command of its own.

Incorrect — ${path@Q} wraps the value, so it stays one word; dash misreads the quoting, which corrupts the value rather than splitting it.

BAn error from dash, %q: invalid directive, so stat never runs and ssh exits 2 for the whole command.

Incorrect — That message is dash's printf lacking %q; ${path@Q} is expanded by the local Bash before ssh sends anything.

CThe exact path, because ssh passes the words after the host to stat directly, without a remote shell.

Incorrect — ssh joins the words into one string for the remote login shell, which parses it again; that shell is dash here.

DOne argument with a changed value: dash reads the $'...' that @Q emitted as a dollar sign and a string.

Correct — It is corruption, not injection; for a POSIX sh, quote with single quotes and '\'', as shlex.quote does.

14A script started with sudo runs exec {key}< /etc/app/signing.key, does its root step, then exec setpriv --reuid="$user" --regid="$gid" --clear-groups --no-new-privs --inh-caps=-all --reset-env -- "$0" --worker. The review says the worker can still read the key. Is that so?

AYes: setpriv keeps open descriptors, so the worker inherits the key's; close it with exec {key}<&- before the exec.

Correct — Changing uid does not close descriptors, and whatever the root part opened stays readable after the drop.

BNo: --reset-env and --clear-groups remove each trace of root, including the files that the root part had opened.

Incorrect — --reset-env clears environment variables and --clear-groups supplementary groups; neither touches open file descriptors.

CNo: the kernel checks the file's permissions again on each read after the uid changes, so the reads fail.

Incorrect — Permissions are checked when a file is opened; an open descriptor keeps the access it was opened with.

DYes, but through /proc/self/environ, where the key's path is still listed after the privilege drop.

Incorrect — The risk is the inherited descriptor itself; the environment was reset, and the key's content was never in it.

15A Bats test starts ./guarded.sh &, runs its assertions, then kills the job. When an assertion before the kill fails, the CI job hangs until the runner's time limit instead of reporting not ok. What is the cause and the fix?

ABats retries a failing test until it passes, and the job's leftover scratch directory makes each retry fail again.

Incorrect — Bats does not retry tests; a failing assertion ends the test at once.

BThe kill line never runs, and Bats waits for SIGTERM to reach each job it started; add trap 'kill 0' EXIT to the test.

Incorrect — Bats does not wait for signals, and kill 0 would signal the process group Bats itself runs in.

CThe job inherited Bats' fd 3, and Bats waits for each holder of it; start the job with 3>&- and stop it in teardown.

Correct — It is the inherited-descriptor bug inside the harness, and teardown runs after a failed test too.

DThe test's wait blocks forever on a background job, so replace it with a sleep placed before the assertions.

Incorrect — The hang happens when the test failed before its kill and wait; the harness waits on a descriptor, not on the test's wait.

16To make healthcheck.sh testable, a teammate proposes CURL=${CURL:-curl} in the script and calling "$CURL", so tests can set CURL=./fakebin/curl. Why does the lesson fake curl with a function, or a fakebin directory put first on PATH for the test command, instead?

ABats cannot pass environment variables to a script started with run, so the variable would not reach the script.

Incorrect — run passes the environment through; the lesson sets FAKE_CODE that way.

BThe variable is a production hook: whoever sets the job's environment, a CI variable or an EnvironmentFile, picks the binary.

Correct — A test function exists in the test shell alone, so the fake adds no switch that production could flip.

CA variable cannot hold a path with a slash in it, so "$CURL" would look for a command that is literally named ./fakebin/curl.

Incorrect — A variable can hold a path, and "$CURL" runs it; the problem is who else can set it.

DShellCheck rejects commands run through a variable, so the format gate would fail on the new line.

Incorrect — ShellCheck accepts "$CURL" args; no gate fails on it.

17An earlier summarize.py anchored its pattern at the end only, Failed password for (?:invalid user )?(?P<user>.*) from (?P<addr>\S+) port \d+ ssh2$, used with search. It meets

... sshd-session[2490]: Failed none for invalid user Failed password for root from 6.6.6.6 port 1 from 203.0.113.45 port 51400 ssh2

. What happens, and what fixes it?

AIt counts a failed password from 203.0.113.45, though sshd logged another failure; anchor the start at sshd's prefix too.

Correct — The username supplied the words and an end-only anchor let search start inside it; anchored at both ends, the lab counted nothing for that line.

BIt credits 6.6.6.6, the address inside the username, because search takes the first from ... port pair that it meets on the line.

Incorrect — The greedy user group runs to the last from ... port ... ssh2, so the address is sshd's 203.0.113.45; the bug is that the line counts at all.

CIt skips the line, because the pattern needs Failed password for and this is a Failed none for line.

Incorrect — search looks anywhere in the line, and the username supplies Failed password for; that is the hole.

DIt raises ValueError from ipaddress, because the address group captures 6.6.6.6 port 1 as one token.

Incorrect — \S+ cannot contain spaces, and the captured address is a valid IP; nothing raises.

18A developer adds print(f"warning: {skipped} lines skipped") to the Python core that logwatch.sh calls. That night the wrapper exits 2 with logwatch: cannot read the report, although the log was fine. Why?

AThe core now returns 1 when it skips lines, and the wrapper treats a non-zero status from it as an error.

Incorrect — print does not change the exit status; the core still returned 0, and the failure came later, in jq.

Bjq refuses reports that contain the word "warning", so the wrapper's filter rejected an otherwise valid report.

Incorrect — jq has no such rule; it failed because the captured text was no longer one JSON document.

CThe here-string <<<"$report" cuts its input at the first newline, so jq saw the warning line alone.

Incorrect — A here-string passes the whole value, newlines included; jq saw the warning first and stopped there.

Dprint writes to stdout, which the wrapper captures as the JSON report, so jq fails on the extra line.

Correct — The contract is data on stdout and messages on stderr; one stray line breaks the parser, and the wrapper correctly refuses to guess.

18 questions · explanations appear as you answer

Python automation engineering

16 questions

01blsync.toml sets max_remove_fraction = 0.2, the unit exports BLSYNC_MAX_REMOVE_FRACTION=0.9, and cron runs blsync --max-remove-fraction 0.5 plan. The feed would remove 3 of the 10 current entries. What happens?

AThe reviewed file wins, since the file is the audited layer, so the plan is refused and the command exits 1.

Incorrect — The file is the second layer, not the last; the environment and then the flag override it.

BThe strictest of the three values applies, 0.2, so removing 3 entries is refused with exit status 1.

Incorrect — The loader does not pick the strictest value; it merges in a fixed order, and the flag's 0.5 is used.

CThe flag wins over the file and the environment; 3 of 10 is within 0.5, so the plan is saved and the command exits 0.

Correct — Layers merge as defaults, file, environment, flags, the later one winning, and the merged value is what gets checked.

DThe environment and the flag disagree, so the loader raises ConfigError and the command exits 78.

Incorrect — Disagreement between layers is normal: later layers override earlier ones, and 78 is for unusable values.

02blsync plan works from a shell in the project directory. As a system service it fails at once, and the journal shows blsync: config: blsync.toml: [Errno 2] No such file or directory: 'blsync.toml' and status=78/CONFIG. What is the fix?

ASet WorkingDirectory= (the StateDirectory= path) and use absolute paths: a service starts in /.

Correct — The default config and plan.json are relative to the working directory, which is / for a system service unless the unit sets one.

BMove the settings into EnvironmentFile=, because a service cannot read a TOML file on its own.

Incorrect — A service can read the files its user can reach; the problem is where the relative path is resolved.

CRun the service as root, because the service user cannot open files that live in a project directory.

Incorrect — The error is ENOENT, not EACCES: the file was looked up in the wrong directory, not refused.

DTreat 78 as transient and let Restart=on-failure retry until the file appears on disk.

Incorrect — 78 means the configuration is unusable; a retry repeats the same lookup in the same wrong directory.

03The plan job uploads plan.json as a CI artifact, and the apply job runs blsync apply on it without --plan-sha256. Someone with write access to the artifact store adds one valid address that stays inside every limit. Which control would have refused the edit?

AThe state digest check, because the edited plan no longer matches the state it was computed from.

Incorrect — The state did not change; the plan's state_sha256 still matches, so this check passes.

BBinding apply to the digest the reviewer recorded, with --plan-sha256 or a signature on the plan.

Correct — Validation catches malformed or out-of-limit plans; a well-formed edit is caught by the digest of what was reviewed.

Ccheck_limits at apply, because an entry that was not in the feed counts against max_changes twice.

Incorrect — One more entry stays inside the limits, so the second run of check_limits passes it.

DPlan.load's type checks, because each address is compared with the feed that produced the plan.

Incorrect — Plan.load checks shape and address syntax, not origin; a valid address passes.

04In ticketer's run(), the except* PermanentError clause sets status = 1 and the except* TransientError clause sets status = status or 75. post_all raises an ExceptionGroup holding one DeadlineExceeded and one PermanentError. What does run() return?

A75, because the last clause that matches decides the status and replaces what the earlier clause set.

Incorrect — The second clause runs, but status or 75 keeps the 1 that the first clause set.

B70, because DeadlineExceeded is neither a PermanentError nor a TransientError and reaches the catch-all for bugs.

Incorrect — DeadlineExceeded subclasses TransientError, so the second clause matches it.

C1, but the deadline failure is never logged, because the group is consumed once one except* clause has run.

Incorrect — Several except* clauses can run for one group; the transient part is logged by its own clause.

D1: both clauses run, since DeadlineExceeded is a TransientError, and status or 75 keeps the 1 the first clause set.

Correct — Each except* clause receives its matching part of the group, and the failure that needs a human outranks the transient one.

05ticketer --budget 60 hung for 40 minutes on one request. A packet capture shows the API's proxy sending the status line and headers one byte every half second. Why did neither the per-call timeout nor the budget stop it?

AThe per-call timeout was larger than the budget, and Deadline.timeout() returns the larger of the two values.

Incorrect — timeout() returns min(cap, remaining), the smaller value; the size of the timeout was not the problem.

BSocket timeouts bound silence, and the clock check begins after the headers; an outside timeout is the backstop.

Correct — Each byte reset the silence timer and the body reader never started, so a limit outside the process had to end it.

Cread_body() checks the clock after each chunk, so the drip should have been caught; the budget must have been off.

Incorrect — read_body runs after urlopen returns, and urlopen reads the status line and headers first.

DPython blocks SIGALRM during socket reads, so the budget's alarm was delivered after the headers ended.

Incorrect — The Deadline class uses the monotonic clock, not SIGALRM; no alarm is involved.

06ticketer derives each idempotency key from the finding's id. A run posted two findings just as the ticket API failed: the tickets were created, but the answers were lost. The API then stayed down for three days, and the next run opened a second ticket for both. The API documents a 24-hour key window. What went wrong?

AThe keys had expired, so the same key was a new request; past the window, the server must refuse duplicate finding ids.

Correct — A stable key replays within the server's memory of it; after a long outage, a server-side duplicate check on the finding id has to catch it.

BThe key should include the run's date, so that a re-run after an outage can be matched to the first attempt.

Incorrect — A date makes each day's key new, which creates duplicates even inside the window.

CThe retry loop ran out of attempts, and after that ticketer stops sending the Idempotency-Key header at all.

Incorrect — The header goes with each attempt of each run; the key was the same, and the server no longer knew it.

DA 503 during the outage cleared the server's key store, which a well-behaved API keeps for 30 days.

Incorrect — The documented window was 24 hours; nothing had to clear it, three days simply outlasted it.

07A log-counting service moves from Ubuntu's python3 to a free-threaded 3.14t build for speed. Four threads update a shared stats.hits += 1. On the old build the totals always matched; now they come out low. What is the diagnosis?

AThe free-threaded build does not keep objects consistent, so shared attributes get corrupted and need queues.

Incorrect — The free-threaded build keeps built-in objects internally consistent; what breaks is a read-modify-write spread over several operations.

B3.14t runs each thread in its own interpreter, so every thread increments its own copy of stats.

Incorrect — Threads in the free-threaded build share one interpreter and its objects; that sharing is why the updates collide.

C+= is read, add, write; the GIL build switched threads at points that happened to spare it. Use a lock.

Correct — The lesson lost updates on the GIL build too once a call sat between the read and the write; a lock fixes both builds.

DThe totals are low because 3.14t is slower on one thread, so some increments had not run when the count was read.

Incorrect — The threads were joined before the count was read; slower single-thread speed changes timing, not the final total.

08A CPU-bound scorer ran 3 times faster with threads on a free-threaded build in testing. In production, with a vendor C extension imported at start-up, threads give no speed-up, and a warning about the GIL appears once in the log. What is the likely cause?

AProduction runs Ubuntu's python3, which accepts PYTHON_GIL=0 but ignores it without an error.

Incorrect — The default build refuses PYTHON_GIL=0 with a fatal error; it does not ignore it silently.

BThreads in the free-threaded build share a single core unless the script pins them with os.sched_setaffinity itself.

Incorrect — The free-threaded build runs threads in parallel on several cores without an affinity call.

CThe free-threaded build switches the GIL back on after a while, to limit its single-threaded overhead.

Incorrect — No such timer exists; the GIL comes back when PYTHON_GIL=1, -X gil=1 or an unmarked extension forces it.

DThe extension is not marked as GIL-free, so importing it turned the GIL back on; check sys._is_gil_enabled().

Correct — Py_GIL_DISABLED says how the interpreter was built; whether the GIL is on now is a run-time fact.

09On Ubuntu's default (GIL) build, which of these jobs gets faster when its work is spread over a ThreadPoolExecutor on a 4-CPU host?

AComputing an entropy score in pure Python for each of 2 million generated host names.

Incorrect — Pure-Python bytecode takes turns on the GIL; the lesson's threads matched serial time.

BRunning hashlib.file_digest(f, "sha256") over thousands of multi-megabyte evidence files kept on local disk.

Correct — Large hash updates and file reads release the GIL, so the threads overlap on several cores.

CCalling hashlib.sha256(token).digest() in a loop over 400,000 tokens of 64 bytes.

Incorrect — Calls on buffers under 2048 bytes keep the GIL, and the loop around them is Python bytecode.

DIncrementing a shared counter a million times, which threads finish in fewer rounds than one thread.

Incorrect — Contended increments do no parallel work and need a lock for a correct total.

10An asyncio job wraps a blocking urlopen call in asyncio.to_thread and starts 10 of them in a TaskGroup. Each request takes 0.2 s, the host has 4 CPUs, and the run takes about 0.42 s rather than 0.2 s. Why?

AThe default executor has min(32, CPUs + 4) workers, 8 here, so ten blocking calls need two rounds.

Correct — to_thread hands work to the loop's default thread pool; to go wider, set a larger default executor or use an async client.

BA TaskGroup starts its tasks one after another, so the ten requests overlap in pairs.

Incorrect — A TaskGroup starts each task at once; the limit is the thread pool the calls are sent to.

Cto_thread holds the GIL for the whole blocking call, so the event loop runs the ten calls in two rounds.

Incorrect — Blocking network waits release the GIL; the rounds come from the size of the pool.

DThe loop blocks for 0.2 s while it starts each worker thread, which adds to the run time.

Incorrect — Thread start-up takes microseconds; two rounds of 0.2 s account for the time.

11An early version of run_probe killed the probe's process group with os.killpg(proc.pid, signal.SIGKILL) in its finally block and then called await proc.wait(). A probe stopped by the output cap hung the whole run. Why does the lesson call await proc.communicate() there instead?

Await() does not reap processes killed by SIGKILL, so it waits for an exit status that never comes.

Incorrect — The kernel reports a SIGKILL death like any other exit; wait was blocked on the pipe, not on the status.

Bkillpg is asynchronous, so wait() ran before the signal arrived and saw the probe still running.

Incorrect — A late signal would still end the probe, and wait would then return; the stall was on the pipe.

Casyncio's wait() also waits for the pipe to close; communicate drains what the capped reader left, then reaps.

Correct — The pipe the capped reader stopped draining never closed on its own; communicate reads the rest and waits for the process.

Dcommunicate kills the group a second time, which is needed because children can escape the first SIGKILL sent to it.

Incorrect — communicate sends no signals; SIGKILL to the group already ended each member.

12ssh_fleet.py connects with timeout=5, banner_timeout=5, auth_timeout=5 and passes timeout=DEADLINE + 5 to exec_command, yet it still wraps the command as timeout 10 sh -c ... on the host. What does the host-side timeout add?

AIt replaces the paramiko timeouts, which apply to the first connection in a thread pool and not to the others.

Incorrect — The paramiko timeouts apply to each connection; they cover different stalls than the remote timeout does.

BIt stops the remote login shell from reading .bashrc, which would otherwise delay the command's start.

Incorrect — timeout does not change how the login shell starts; it limits how long the command runs.

CA clean exit status 143 in place of paramiko's socket.timeout, which cannot be caught inside a thread pool worker.

Incorrect — timeout exits 124 when it stops a command, and paramiko's timeouts can be caught; that is not the reason.

DA run-time limit: paramiko's timeouts bound silence, and a command that keeps printing slowly does not trip them.

Correct — Connect, banner, auth and read timeouts end a stall; a limit on duration has to sit next to the command.

13A nightly fleet run prints 10.0.4.21:22: ERROR host key mismatch (ssh-ed25519) for one host and exits 1. The host was rebuilt yesterday. Which response keeps the fleet's host-key policy intact?

ARun ssh-keyscan against the host, append its answer to known_hosts, then rerun the fleet job.

Incorrect — That trusts whatever answers on the network right now, which is trust on first use done by hand.

BTake the new key from the provisioning inventory, regenerate known_hosts from it, and rerun the job.

Correct — Rotation goes through the inventory, not through first contact; until then the unknown key keeps failing closed.

CSwitch that host to WarningPolicy until the next rebuild, so the run logs the mismatch and carries on.

Incorrect — WarningPolicy logs and connects, handing the session to whoever presented the key.

DDelete the host's line from known_hosts, so RejectPolicy treats it as a new host and records its key.

Incorrect — RejectPolicy refuses unknown keys and records nothing, so the run keeps failing, correctly.

14A threaded ingest service must run regular expressions from a rule file it cannot rewrite, so it wraps each match in signal.signal(signal.SIGALRM, handler) and signal.alarm(1). At start-up the worker threads fail with ValueError: signal only works in main thread of the main interpreter. What should it use?

AA self-bounding engine: regex with timeout=, which works in worker threads, or google-re2, which is linear.

Correct — Both bound the work per match without signals; compare their answers with re on tricky inputs before swapping.

Bsignal.setitimer() in place of alarm(), which works in worker threads and also allows timers shorter than one second.

Incorrect — setitimer has finer steps, but the handler still has to be installed from the main thread.

CA length cap of 253 characters on each field, which keeps the run time of each pattern small.

Incorrect — A cap bounds input length, not work: the lesson's exponential pattern took over a second on 25 characters.

Dre.compile with re.ASCII, which turns off the backtracking that makes nested patterns slow.

Incorrect — re.ASCII narrows \w and its relatives; it does nothing about backtracking.

15A record loader runs raw = json.loads(text) inside try, with except json.JSONDecodeError: reject(text), and then reads raw["target"]. Which input crashes the loader instead of being rejected?

A{"target": "web01", a record whose closing brace was cut off somewhere in transit.

Incorrect — A truncated object raises JSONDecodeError, which the handler catches.

B{"target": "web01;id"}, a host name that carries a shell metacharacter.

Incorrect — That parses fine and reaches the grammar check, which is the next step's job.

CAn empty string, which json.loads quietly turns into None instead of raising an error.

Incorrect — json.loads("") raises JSONDecodeError, so the handler rejects it.

DA 5,000-digit number, which raises a plain ValueError from the integer digit limit.

Correct — Deep nesting escapes too, with RecursionError, and [1] parses but is not an object; catch (ValueError, RecursionError) and check the shape.

16A JSON log formatter runs payload.update(record.fields) after it has set ts, level and msg. A record parsed from an upload carries {"user": "bob", "level": "debug", "msg": "login ok"}. What does the log line show?

AThe upload's values escaped as data inside the real ones, because json.dumps quotes each value it writes.

Incorrect — json.dumps escapes characters, but a merged key has already replaced the real one before the dump.

BTwo JSON lines, one of them forged, because duplicate keys make json.dumps split the record.

Incorrect — A dict has one value per key and json.dumps writes one object; no second line appears.

CA well-formed line whose level and msg are the upload's; nest the fields or prefix clashing keys.

Correct — json.dumps stops line forging, but merging lets input overwrite the reserved keys inside a valid line.

DA KeyError at format time, because update refuses keys that already exist in the dict.

Incorrect — dict.update overwrites existing keys without complaint.

16 questions · explanations appear as you answer

Operate and ship

9 questions

01sweep runs hourly from cron through sweep-cron.sh, and nobody has been mailed for a week. The metric file shows sweep_last_run_seconds from 20 minutes ago, sweep_last_success_seconds from 8 days ago and sweep_last_exit_code 75. What is going on?

AIt runs and keeps failing transiently; the wrapper stays quiet on 75, so the stale last-success is the alarm.

Correct — 75 means "a later run can finish", and a week of it means it will not; the freshness gate turns that into a page.

BThe job has stopped running, so the crontab entry and the cron daemon need checking before anyone reads the logs.

Incorrect — A job that stopped would show an old last-run time; this one ran 20 minutes ago.

CThe metric writer is broken, since last-success cannot be older than last-run on a working job.

Incorrect — A failed run moves last-run and keeps last-success; the gap is the designed signal.

Dnode_exporter cannot read the file, so it serves the values it cached at the last good scrape.

Incorrect — node_exporter keeps no such cache; an unreadable file makes the series disappear.

02In sweep, the first target is under a change freeze (a PermanentError, recorded in the report), and the second target's state file is truncated JSON. What exit status does the run end with?

A1, because a permanent failure was already recorded, and the most demanding recorded failure kind decides.

Incorrect — The crash comes before outcome() runs; except Exception sets 70, and the report's contents are not ranked.

B70: the JSONDecodeError fits no rule, so the crash handler logs a traceback and ends the run as a bug.

Correct — An unclassified exception means a state nobody planned for, and 70 outranks 1 and 75.

C75, because a file that cannot be parsed now may be complete by the next run.

Incorrect — A JSONDecodeError is not classified as transient; no rule covers it, so it is treated as a bug.

D78, because the fleet directory named in the config contains a file the job cannot use.

Incorrect — 78 is for an unusable configuration before anything was attempted; this file is target state.

03After an incident, check_audit.py audit/sweep.jsonl prints chain over 9 records: intact, but the collector that receives each line as it is written holds 13. What does that show?

ANothing is wrong: the collector counted some records twice, because start and done lines share a target.

Incorrect — Each event is its own line; start and done are two records in both copies.

BFour lines were edited, and verify() missed the change because sort_keys=True hides edited values.

Incorrect — An edited line with a successor breaks the chain; the local file verified because nothing followed the missing lines.

CThe flock was missing, so two writers chained to the same line and four records were overwritten.

Incorrect — Concurrent appends without the lock produce a broken chain, not a shorter intact one.

DThe last 4 lines were cut locally; a hash chain cannot see a missing tail, hence the off-host copy.

Correct — Removing lines from the end leaves a shorter valid chain; the off-host copy is the evidence.

04Servers get certcheck with uv tool install certcheck-0.1.0-py3-none-any.whl, and the project commits a uv.lock that pins cryptography==50.0.1. After a new cryptography release, a server installed that week behaves differently from staging. Why?

AThe wheel embeds its dependencies at build time, so the server kept an older copy of cryptography.

Incorrect — A py3-none-any wheel carries the project's modules alone; dependencies are installed next to it.

Buv.lock pins versions for the laptop's platform, so Linux servers resolve their own versions on install.

Incorrect — uv resolves one lock for each platform the project supports; the server skipped the lock entirely.

Cuv tool install resolved the wheel's cryptography>=50 range on the day and ignored uv.lock.

Correct — A tool install matches the lock while nothing newer exists; install servers from the hashed lock.

Duv tool install builds from the sdist, so cryptography was compiled on the server with other flags.

Incorrect — The install used the wheel you named and cryptography's published wheels; nothing was compiled.

05A single-file certcheck.pyz, built with python3 -m zipapp from a directory holding certcheck and its hash-locked dependencies, fails on start with ImportError: cannot import name 'x509' from 'cryptography.hazmat.bindings._rust' (unknown location). What is the cause, and what is a sound choice?

AThe lock was exported for another CPU, so the archive holds the wrong wheel; rebuild it on the target host.

Incorrect — The same archive fails on its own build host; the lesson built and ran it on one machine.

BPython cannot load an extension module from inside a zip; ship the wheel and its lock, or a container.

Correct — Zip imports cover source and bytecode; a tool with compiled dependencies needs an unpacked install, pex's cache or an image.

Czipapp strips .so files to keep archives portable; pass --compress so that they are kept.

Incorrect — zipapp does not strip files; the extension is in the archive and cannot be loaded from there.

DThe archive lacks a __main__.py that raises SystemExit, so the imports run in the wrong order.

Incorrect — A custom __main__.py fixes the exit status, not imports; this is about loading compiled code.

06uv export --format pylock.toml -o pylock.toml followed by pip install -r pylock.toml stops with The editable requirement file:///srv/certcheck/ (from pylock.toml) cannot be installed when requiring hashes. What is the fix?

AExport with --no-emit-project and install the project from its built wheel in a second step.

Correct — A hashed lock puts pip in hash-checking mode, and a directory has no single file to hash.

BAdd --no-deps to the pip command so that pip skips hash checking for the project entry.

Incorrect — --no-deps stops dependency resolution; it does not turn off hash-checking mode.

CDrop the hashes from the export with --no-hashes, since pylock support in pip is experimental anyway.

Incorrect — That removes the verification the lock exists for, to work around a problem with one entry.

DRun pip install -e . first, so the editable entry is already satisfied when pip reads the lock.

Incorrect — pip still refuses the editable entry in hash mode; the project has to stay out of the hashed lock.

07certcheck shipped with 100% branch coverage, yet a bug slipped through: expiring certificates no longer set exit status 1. The test for that path calls main() on an expiring certificate and asserts nothing. What does this show?

AThe coverage floor was too low; raising fail_under to 100 would have failed the build.

Incorrect — Coverage was already 100%; the path ran, so no floor would have noticed.

BBranch coverage counts if statements alone, so the value main() returned was never measured by the tool.

Incorrect — Branch coverage adds both sides of each branch to line coverage; the lines ran and were counted.

Cpytest-cov skips functions that return an int, so main() was not measured at all.

Incorrect — There is no such rule; cli.py's lines were measured, as the report showed.

DCoverage records which lines and branches ran, not whether a test checked the result; review the asserts.

Correct — A call without an assertion covers the same lines as a real test; the floor finds untested code, reviews find unchecked code.

08A pull request changes one workflow line to uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v7.0.1, a different full SHA with the same comment, and actionlint passes. Why can this still be a problem, and what catches it?

AIt cannot be: a 40-character SHA names immutable code from the upstream repository, so review can stop there.

Incorrect — A SHA is immutable, but GitHub serves commits from any fork under the upstream name, and this one is v6.0.3's.

Bactionlint rejects unknown SHAs when it runs online, so its pass means the commit is upstream's v7.0.1.

Incorrect — actionlint checks syntax and expressions; it does not resolve tags or check pins.

CThe SHA may be another release or a fork's commit; resolve the tag upstream with git ls-remote and compare.

Correct — check-pins.sh asks upstream which commit the tag names and reports MISMATCH; a format check cannot.

DDependabot would revert the change within a day, so the risk is limited to one day of runs.

Incorrect — Update bots propose new pins; they do not audit whether an existing SHA matches its comment.

09Deploys run cosign verify --key cosign.pub "$ref" on a digest reference. A test build signed months ago with the same release key is deployed to production by mistake, and verification passes. What would have refused it?

AVerifying by tag instead of by digest, since test builds are pushed under their own test tags.

Incorrect — Tags can be moved to any digest; switching to them weakens the check instead of narrowing it.

BSigning releases with an annotation such as -a env=prod, and requiring it with cosign verify -a env=prod.

Correct — A key signature says the key signed that digest, so each image it ever signed passes; the annotation narrows that to releases.

CA transparency-log check, which rejects signatures made before the key's most recent rotation.

Incorrect — The log records when a signature was made; it does not rank test signatures below release ones.

DAn SBOM attestation, because a test build's SBOM lists different packages from the release build of the same code.

Incorrect — An attestation says what is inside; it does not say the build was meant for production.

9 questions · explanations appear as you answer

Capstone

4 questions

01A reviewer wants certrun_last_success_seconds to move on exit 0 alone, as sweep's did in "Observable jobs". admin has waited for a person for three weeks, so each apply exits 1, and the alert is time() - certrun_last_success_seconds > 2 days. What would the change do?

ANothing visible: exit 1 moves no metric today either, because certrun.sh writes its metrics after a clean run and no other.

Incorrect — certrun.sh writes the run metrics on each apply that gets the lock, 77 and 143 included, and last-success already moves on exit 1.

BMake the alert sharper, because an exit 1 run did not handle the endpoints it was asked to look at.

Incorrect — Exit 1 means every endpoint was handled and some wait for a person; certrun_needs_attention already pages for those.

CFire the alert for weeks: last-success would stay old while admin waits, hiding a stopped timer behind it.

Correct — With 1 as a normal state, last-success means "a run handled every endpoint"; its age is the stopped-timer alert, and needs_attention pages for admin.

DBreak the textfile collector, because node_exporter drops a gauge that has not changed for more than a day.

Incorrect — node_exporter serves whatever value the file holds; a stale value is exactly what the age alert measures.

02A timer run of scr-capstone-certrun@lab.service ends 40 seconds into its 120-second budget. The journal shows the core was killed with SIGKILL before the deadline (out of memory?), then status=70/SOFTWARE, and the unit is not restarted. An operator wants that case mapped to 75 like a timeout. What is the right answer?

AKeep 70: a SIGKILL before the deadline came from something else, such as the OOM killer, and a retry would repeat it.

Correct — certrun.sh calls 137 a deadline when the deadline has passed; before that it is a failure a person must see, and 70 is in RestartPreventExitStatus.

BMap it to 75, because timeout -k sends SIGKILL, so a 137 from the core is the deadline doing its job.

Incorrect — timeout's SIGKILL comes after the budget runs out; 40 seconds into 120 it could not have been timeout.

CMap it to 143, because the core died of a signal and the unit treats a death by signal as a clean stop.

Incorrect — SuccessExitStatus lists SIGTERM alone; a SIGKILL was not a requested stop, and 143 would misreport it.

DKeep 70, but expect a restart anyway, because Restart=on-failure retries any non-zero status that a oneshot unit returns.

Incorrect — RestartPreventExitStatus=1 2 70 77 78 prevents that; 70 waits for a person, by design.

03After a run rolled back shop (the new certificate came from an untrusted issuer), each later apply prints a deploy was rolled back (...); fix the cause, then delete pending/shop.json, and the CA's counters do not move. A colleague proposes a daily cron job that deletes rolled-back records so rotations retry by themselves. What does that lose?

ANothing: the record holds no private key and exists so that the report can show a reason for the failure.

Incorrect — The record holds the new private key and the exact request bytes, and it blocks further CA requests.

BThe lock: without the record, two runs could rotate shop at the same time and deploy two different certificates to it.

Incorrect — The flock on DESTINATION.lock prevents overlapping runs; the record is about the rotation, not concurrency.

CThe backup: deleting the record also deletes the old certificate file that the rollback put back in place.

Incorrect — The rollback renamed the backup into place on the host; the record in state/ is a separate file.

DThe stop for a person: each deletion starts a new rotation and CA request, and an unfixed cause rolls back again.

Correct — The record is marked rolled back so no later run asks the CA again; a person confirms the cause is gone, then deletes it, as the Try this does.

04To let the job write a debug log next to its code, a teammate changes /opt/scr-capstone to certrun:certrun, mode 0755. release/, certrun.sh, verify_core.py and etc/cosign.pub stay root-owned. Does the verify gate still stop a core that nobody signed?

AYes: each file that decides what runs is still root's, so the account can read those files and change none.

Incorrect — Owning the files is not enough; the account now owns the directory that holds them.

BYes, since cosign checks the wheel against the pinned public key, and the service account cannot forge a signature.

Incorrect — The account can put its own key, its own gate and its own release in place; the check then passes on its terms.

CNo: owning the parent lets the account rename and replace release/, certrun.sh and etc/, swapping the gate.

Correct — Renaming needs write access to the parent directory, as the lab's refused mv showed; root's all the way up keeps the gate out of the account's reach.

DNo, but because the debug log could fill the disk and make mktemp fail, which would turn each run into a 70.

Incorrect — A full disk is a separate problem; the ownership change hands the account control over what runs.

4 questions · explanations appear as you answer