Python for Bash users: values, collections and loops

Strings, numbers, lists, dicts, sets, loops and f-strings, mapped from Bash.

You already write Bash. This lesson gives you the Python the rest of the course is written in: values and their types, lists, tuples, dictionaries and sets, loops, comprehensions, f-strings, truth tests, and the difference between text and bytes. Each idea is mapped to the Bash construct you already know, and every example works on things an operator handles: ports, hosts, log lines and file sizes. You will also read your first tracebacks, the report Python prints when a program stops on an error. The lab works in ~/ps-basics: unpack the lesson files in your home directory to get every file below, or create the directory and the files as they appear.

Values have types

In Bash every variable holds a string, and the context decides how it is used: $(( )) treats it as a number, [[ ]] compares it as text. In Python the value itself carries a type. The ones you meet first are str (text), int (whole numbers), float (numbers with a fraction), bool (True or False) and None, which means "no value" and plays the role an unset or empty variable plays in Bash. The type decides what an operator does.

raw = "8443"  # text from a config file, argv or input() is always a str
print(type(raw), raw + "1")
port = int(raw)
print(type(port), port + 1)
print(port / 2, port // 2, port > 1024)
print("listening on port " + port)

Read it line by line. raw + "1" joined two pieces of text into 84431, the same as "${raw}1" in Bash. int(raw) converted the text into a number, and now + 1 is arithmetic, like $((raw + 1)). / always gives a float (4221.5); // divides and drops the fraction, like Bash's $((8443 / 2)) for positive numbers (for negative ones Python rounds down and Bash rounds towards zero). port > 1024 produced a bool. Then the last line stopped the program.

That block is a traceback. Read it from the bottom up. The last line is the exception type and its message: TypeError, you cannot add an int to a str. Above it are the file, the line number and the source line, and Python 3.14 underlines the failing operation: the ^ sits under the +. "Most recent call last" means the place where it failed is at the bottom. The program exited with status 1. (In your own terminal Python 3.14 colours the traceback; the text is the same.)

Python refuses to guess whether you meant text or arithmetic. When you want text, use an f-string: a string with an f in front, where any expression in braces is evaluated and converted to text.

int() on text that is not a number raises a ValueError instead. The next lesson shows how to catch it and report the bad input.

Lists, tuples and loops

A list is Python's version of a Bash indexed array. hosts=(web01 web02 db01) becomes ["web01", "web02", "db01"], hosts+=(cache01) becomes hosts.append("cache01"), ${#hosts[@]} becomes len(hosts) and ${hosts[-1]} becomes hosts[-1]. A slice, hosts[1:3], takes items 1 and 2: the start is included and the end is not.

hosts.append(...) is a method call. Values come with their own functions, called methods, which you call as value.method(args), and each type has its own set: help(str) lists the ones for text. The ones this course uses most are strip, split, lower, startswith and join for strings, and append, pop, get and items for lists and dicts.

hosts = ["web01", "web02", "db01"]
hosts.append("cache01")
print(len(hosts), hosts[0], hosts[-1], hosts[1:3])
 
for host in hosts:
    if host.startswith("db"):
        tier = "database"
    elif host.startswith("cache"):
        tier = "cache"
    else:
        tier = "web"
    print(host, "->", tier)
 
pending = ["web02", "db01"]
while pending:  # an empty list counts as false, so the loop stops when it drains
    host = pending.pop(0)
    print("restarting agent on", host)
 
disks = [("/", 57), ("/var/log", 91), ("/home", 34)]
for mount, used in disks:
    if used >= 90:
        print("ALERT", mount, used)

There is no do, done or fi. A line ending in a colon opens a block, and the block is everything indented under it (four spaces is the convention). The indentation is the syntax, so a line indented differently belongs to a different block. if/elif/else works like Bash's, and host.startswith("db") is the test [[ $host == db* ]] would do.

while pending: keeps looping as long as the list has items, and pending.pop(0) removes and returns the first one, so the loop ends when the queue is empty. The disk list holds tuples: ("/var/log", 91) is a fixed pair that cannot be changed after it is made, a good fit for a record such as a mount point and its usage. for mount, used in disks unpacks each pair into two names, the job read -r mount used does in a Bash loop.

Dictionaries and sets

A dictionary (dict) maps keys to values, like declare -A in Bash, and counting things per key is its everyday job. One difference bites Bash users at once. ${fails[$ip]} for a key that was never set expands to an empty string. Python raises an error instead:

The last line names the exception, KeyError, and the key that was missing. fails["203.0.113.42"] += 1 has to read the old value before it can add 1, and there is no old value. fails.get(ip, 0) returns the value if the key exists and 0 if it does not, which plays the part of ${fails[$ip]:-0}.

Here is a failed-login counter written both ways. The Bash version uses the associative array technique from the Bash course; the Python version does the same work. Both read this sample log on standard input:

2026-09-27T08:14:02.118244+00:00 web01 sshd-session[2201]: Failed password for root from 203.0.113.42 port 51122 ssh2
2026-09-27T08:14:05.402911+00:00 web01 sshd-session[2201]: Failed password for root from 203.0.113.42 port 51130 ssh2
2026-09-27T08:14:40.771403+00:00 web01 sshd-session[2215]: Failed password for invalid user admin from 203.0.113.9 port 40110 ssh2
2026-09-27T08:15:12.004187+00:00 web01 sshd-session[2230]: Accepted password for deploy from 198.51.100.7 port 40022 ssh2
2026-09-27T08:16:55.310522+00:00 web01 sshd-session[2244]: Failed password for deploy from 198.51.100.7 port 40031 ssh2
2026-09-27T08:17:03.829940+00:00 web01 sshd-session[2251]: Failed password for invalid user oracle from 203.0.113.42 port 51188 ssh2
2026-09-27T08:18:21.556071+00:00 web01 sshd-session[2263]: Failed password for invalid user test from 192.0.2.77 port 60001 ssh2
2026-09-27T08:18:24.090315+00:00 web01 sshd-session[2263]: Failed password for invalid user test from 192.0.2.77 port 60002 ssh2

The Bash lesson "Arrays and parameter expansion" showed why a failed-login line cannot be trusted as a whole: sshd logs whatever user name the client sent. Add the same kind of crafted line, with the user name x from 198.51.100.7 port 22. printf "%s" writes it without a newline at the end, as a log cut off mid-write would be, so wc -l, which counts newline characters, still says 8:

Taking the word after the first from now gives 198.51.100.7, one of your own jump hosts in the script below: an address the attacker chose. The address sshd wrote comes after the last from :

#!/usr/bin/env bash
# Count failed SSH logins per source address; the log arrives on stdin.
set -euo pipefail
declare -A fails=()
# || [[ -n $line ]]: also count a last line that has no newline at the end
while IFS= read -r line || [[ -n $line ]]; do
  [[ $line == *"Failed password"* ]] || continue
  addr=${line##* from } # everything after the LAST " from "
  ip=${addr%% *}
  fails[$ip]=$((${fails[$ip]:-0} + 1))
done
for ip in "${!fails[@]}"; do
  printf '%s %d\n' "$ip" "${fails[$ip]}"
done

import sys
 
fails = {}
for line in sys.stdin:
    if "Failed password" not in line:
        continue
    # The user name is whatever the client sent and may contain " from " itself,
    # so take the address after the LAST " from ": the one sshd wrote.
    ip = line.rsplit(" from ", 1)[1].split()[0]
    fails[ip] = fails.get(ip, 0) + 1
 
known = {"198.51.100.7", "198.51.100.8"}  # our own jump hosts
for ip, count in fails.items():
    status = "known" if ip in known else "UNKNOWN"
    print(ip, count, status)
print("unknown sources:", sorted(set(fails) - known))

The Bash version takes the last from with ${line##* from }, as in the Bash course's arrays lesson, and needs || [[ -n $line ]]: read fills line but returns a non-zero status on a last line without a newline, and without that test the loop drops the crafted line. Python's loop has neither problem. import sys makes the standard library module sys available; the next lesson explains modules. for line in sys.stdin is while IFS= read -r line without the traps: no field splitting, no backslash handling, no lost last line, and line keeps its trailing newline when it has one. "Failed password" not in line is the substring test [[ $line == *"Failed password"* ]]. line.rsplit(" from ", 1) splits at the last from only (the r means from the right), [1] is the part after it, and .split()[0] its first word, the address. "known" if ip in known else "UNKNOWN" is a conditional expression: one value or the other, chosen by the test in the middle.

Compare the order. Bash printed the addresses in hash order. A Python dict keeps the order in which keys were first inserted, so the report follows the log. known is a set: a collection of unique values with no order, where ip in known is a fast membership test. set(fails) - known is set difference, every address that failed and is not one of your jump hosts. The crafted line counts against 203.0.113.66, not against your jump host. sorted() returns a new sorted list. It sorts these addresses as text, character by character, which is why 203.0.113.42 lands before 203.0.113.9; sorting addresses as numbers needs the ipaddress module from the regex lesson.

Comprehensions, formatting and truth tests

files = [
    ("auth.log", 18_342_112),
    ("syslog", 4_210_455),
    ("kern.log", 912),
    ("nginx/access.log", 1_204_331_001),
]
big = [name for name, size in files if size > 10_000_000]
print(big)
mib = {name: size / 1024**2 for name, size in files}
print(mib["kern.log"])
 
print(f"{'file':<18} {'bytes':>15} {'MiB':>9}")
for name, size in files:
    print(f"{name:<18} {size:>15,} {mib[name]:>9.1f}")
total = sum([size for name, size in files])
print(f"{len(files)} files, {total / 1024**3:.2f} GiB in total")

[name for name, size in files if size > 10_000_000] is a list comprehension: a loop, a filter and an append in one expression. Read it as "the name, for each name and size in files, where the size is over 10 MB". The version in braces builds a dict the same way. The underscores in 10_000_000 only make the number readable, and 1024**2 is 1024 squared. Keep a comprehension to one loop and one condition; anything longer reads better as a plain for loop.

print(mib["kern.log"]) shows why formatting exists: a raw float prints every digit it has. In an f-string, the part after the colon is a format spec, much like printf: <18 pads to 18 columns aligned left, >15 aligns right, , adds thousands separators and .1f rounds to one decimal. The header uses the same specs on plain strings, so the columns line up.

Truth tests need the same care as types. if value: does not ask whether something is True; it asks whether it is empty:

import os
 
for value in ["", "0", "false", 0, None, [], {}, "web01 "]:
    print(f"{value!r:>8} is {bool(value)}")
 
apply = os.environ.get("APPLY", "")
if apply:
    print("APPLY =", repr(apply), "-> applying firewall changes")
else:
    print("APPLY is empty -> dry run only")

Empty text, zero, None, and an empty list or dict are false. Everything else is true, including the strings "0" and "false". !r in an f-string prints a value's representation, with quotes, which is how 'web01 ' reveals a trailing space that a plain print hides. The last line is the bug: environment variables and command-line arguments are always strings, so APPLY=false is a non-empty string and the script "applies firewall changes". Bash has the same trap with [[ -n $APPLY ]], but there you can at least see that you are testing for non-empty. Compare the value you accept explicitly:

For a switch that changes systems, make the safe behaviour the default and require an explicit value to do anything else. The argparse lesson builds such flags properly.

Text and bytes

A str is text: a sequence of Unicode characters. A bytes value is raw data, and it is what files opened in binary mode, network sockets, command output and hash functions work with. .encode("utf-8") turns text into bytes and .decode("utf-8") turns bytes back into text. Bash never makes you choose; its strings are bytes and the locale decides how they are shown.

import hashlib
 
user = "josé"
data = user.encode("utf-8")  # str -> bytes; files, sockets and hashes work in bytes
print(len(user), len(data), data)
print(data.decode("utf-8"))
print(hashlib.sha256(data).hexdigest()[:16])
print(hashlib.sha256(user).hexdigest()[:16])

josé is 4 characters but 5 bytes, because UTF-8 stores é as the two bytes \xc3\xa9; the b'...' prefix marks a bytes value. Decoding gives the text back. hashlib hashes bytes, so passing the str raises TypeError: Strings must be encoded before hashing, and the working line above it shows the fix. The habit that avoids most encoding bugs: decode data into text where it enters your program, work with str inside, and encode where it leaves. The files and subprocess lessons show where Python does this for you and how to control it.

Try this

Write top_unknown.py in ~/ps-basics. Like count_fails.py, it reads the log on standard input, but it prints only addresses that are not in known and failed at least twice, highest count first, as f"{ip:<15} {count:>3}". Build a list of (count, ip) tuples with a comprehension and sort it with sorted(rows, reverse=True) (tuples compare by their first item, then the next). Break it on purpose: replace fails[ip] = fails.get(ip, 0) + 1 with fails[ip] += 1 and read the traceback (which line, which key). Fix it and verify: on the log with the crafted line, python3 top_unknown.py < auth-sample.log prints exactly two lines, 203.0.113.42 with 3, then 192.0.2.77 with 2.

Takeaway

Convert input to the type you mean at the moment it enters your program: int() for ports and counts, an explicit comparison for yes/no switches, .decode() for bytes. When a type assumption is wrong, the last line of the traceback names it and the lines above point to where.

Related