Structured data: JSON, YAML & CSV

Parse and emit the formats tools speak.

Security tools do not talk to each other in plain English. A vulnerability scanner finishes a run and drops a file on disk. A cloud command lists every storage bucket in your account. A firewall exports its rule set. Every one of those handoffs is written in one of three text formats, and nearly every tool you will ever automate speaks at least one of them. Once you can read and write all three from Python, you can wire almost any two tools together with a script that fits on one screen.

Three formats, three jobs

The three are JSON, YAML, and CSV. Each is good at a different job, which is why all three have survived side by side for years. JSON (JavaScript Object Notation, a way of writing nested data as plain text) is what one machine hands to another: exact, unambiguous, a little noisy for human eyes. YAML (which officially stands for 'YAML Ain't Markup Language', a config format built to be pleasant to type by hand) is where people write settings, so Kubernetes manifests, CI (Continuous Integration, the automated build-and-test step of a pipeline) files, and tool configs all live in it. CSV (Comma-Separated Values, a plain-text spreadsheet) is a grid of rows and columns that opens in Excel or Google Sheets with no fuss. Your work as the glue between tools is almost always the same shape: read one format, reshape the data, write another.

JSON: the machine handoff

A JSON document is a set of labeled boxes, and each box can hold more labeled boxes inside it. A box with named slots is an object, and Python hands it to you as a dict (dictionary, a lookup table that maps names to values). A box holding an ordered pile of items is an array, and Python gives you a list. Text, numbers, true and false, and null fill in the rest. Because every box marks exactly where it starts and stops, a scanner can pack a finding, its score, and its fix version one inside another with zero guesswork about which value belongs to what.

The json module gives you two ways in and two ways out. json.loads takes a string of JSON and returns Python objects (the trailing 's' stands for 'string'). json.load does the same from an open file. Going the other way, json.dumps turns Python objects back into a JSON string, and its indent argument pretty-prints the result with line breaks and spacing so a person can read it. Here is the whole round trip in a few lines.

import json
 
# A scanner hands you findings as JSON text. Here is one, as a string.
raw = '{"id": "CVE-2024-3094", "severity": "CRITICAL", "cvss": {"score": 10.0}}'
 
finding = json.loads(raw)              # parse the string into Python objects
print(type(finding))                   # a JSON object becomes a dict
print(finding["severity"])             # read a top-level field
print(finding["cvss"]["score"])        # reach into the nested object
print(json.dumps(finding, indent=2))   # turn it back into readable JSON text

Two things are worth catching here. Reaching a nested value is a chain of square brackets: finding["cvss"]["score"] steps into the inner object and pulls the number out. And if you ask for a key that is not there, Python raises a KeyError and stops dead. That is a feature during triage, because a silently missing field is exactly how a real finding slips past you unnoticed. When a field is genuinely optional, reach for finding.get("fixed_in") instead, which returns None rather than crashing.

A real scanner report on disk

{
  "scanner": "trivy",
  "scanned_at": "2026-07-17T09:14:22Z",
  "image": "registry.internal/api-gateway:1.8.3",
  "findings": [
    { "id": "CVE-2024-3094", "package": "xz-utils", "installed": "5.6.0",
      "fixed_in": "5.6.2", "severity": "CRITICAL",
      "cvss": { "score": 10.0, "vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N" } },
    { "id": "CVE-2023-44487", "package": "nghttp2", "installed": "1.51.0",
      "fixed_in": "1.57.0", "severity": "HIGH",
      "cvss": { "score": 7.5, "vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N" } },
    { "id": "CVE-2022-40897", "package": "setuptools", "installed": "65.5.0",
      "fixed_in": "65.5.1", "severity": "MEDIUM",
      "cvss": { "score": 5.9, "vector": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N" } },
    { "id": "CVE-2021-33574", "package": "glibc", "installed": "2.31",
      "fixed_in": "", "severity": "HIGH",
      "cvss": { "score": 7.5, "vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N" } },
    { "id": "CVE-2024-2511", "package": "openssl", "installed": "3.0.13",
      "fixed_in": "3.0.14", "severity": "LOW",
      "cvss": { "score": 3.7, "vector": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N" } }
  ]
}

That is trimmed output from an image scan, the kind Trivy or Grype writes. The top level is an object describing the scan, and findings holds a list of five vulnerabilities. Each one carries an id (a CVE, short for Common Vulnerabilities and Exposures, the public catalog that stamps every known bug with a unique number), the package it lives in, the installed and fixed versions, a severity label, and a nested cvss object holding the numeric CVSS (Common Vulnerability Scoring System, a 0.0 to 10.0 danger rating) score. Five findings you can read by eye. Five thousand you cannot, which is where a script earns its keep.

CSV: a spreadsheet in plain text

A CSV file is rows of values with commas between them and a header row naming the columns, and that is the whole idea. The people who receive your security reports, a manager, an auditor, a ticket queue, mostly want a flat table they can sort and filter, not nested JSON. Python's csv module ships two tools built for exactly this. csv.DictReader reads each row into a dict keyed by the header names, so you write row["severity"] instead of counting comma positions. csv.DictWriter does the reverse, taking one dict per row plus a fieldnames list that pins the column order. Build one habit now: open CSV files with newline="". On some systems the csv module and the operating system each add their own line ending, and you get a blank row between every line of real data. It is the single most common CSV bug, and that one argument is the whole fix.

Filter the JSON, write a CSV summary

Here is the everyday task those pieces add up to. Read the scanner's JSON, keep only the findings that matter tonight, sort them worst first by the nested score, and write a clean CSV a human can open.

import csv
import json
 
# 1. Read the scanner's JSON report straight from disk (load, not loads).
with open("findings.json") as f:
    report = json.load(f)
 
# 2. Keep only what an on-call human needs to see tonight.
urgent = [
    f for f in report["findings"]
    if f["severity"] in ("CRITICAL", "HIGH")
]
 
# 3. Sort worst-first by the nested CVSS score.
urgent.sort(key=lambda f: f["cvss"]["score"], reverse=True)
 
# 4. Write a flat CSV a manager can open in a spreadsheet.
with open("summary.csv", "w", newline="") as f:
    writer = csv.DictWriter(
        f,
        fieldnames=["id", "package", "severity", "score", "fix"],
    )
    writer.writeheader()
    for finding in urgent:
        writer.writerow({
            "id": finding["id"],
            "package": finding["package"],
            "severity": finding["severity"],
            "score": finding["cvss"]["score"],
            "fix": finding["fixed_in"] or "no fix yet",
        })
 
print(f"{len(urgent)} of {len(report['findings'])} findings need action")
print(f"top risk: {urgent[0]['id']} ({urgent[0]['package']})")

The filter is one line: a list comprehension that keeps a finding only when its severity is CRITICAL or HIGH. The sort reads the nested cvss score through a small key function, so the worst risk lands on top. writeheader() prints the column names once, then each writerow flattens one finding into the five columns you chose. The finding["fixed_in"] or "no fix yet" trick handles the glibc row, whose fix field is an empty string: an empty string counts as false in Python, so the or hands back the friendly label instead. Reading the file back proves it came out well formed.

import csv
 
with open("summary.csv", newline="") as f:
    reader = csv.DictReader(f)
    for row in reader:
        print(f"{row['severity']:8} {row['id']:16} fix -> {row['fix']}")

CRITICAL CVE-2024-3094    fix -> 5.6.2
HIGH     CVE-2023-44487   fix -> 1.57.0
HIGH     CVE-2021-33574   fix -> no fix yet

DictReader turned the header row into keys on its own, so the reading code never has to know which column sits where. Add a column to the writer later and this reader keeps working untouched.

YAML: friendly config with a loaded gun

YAML is the format people edit by hand, so it drops the braces and quotes that JSON insists on and uses indentation to show nesting, the way an outline uses indentation under each heading. You have already read thousands of lines of it if you have touched Kubernetes, GitHub Actions, or Ansible. To read it in Python you install PyYAML (pip install pyyaml, where pip is Python's package installer) and call one function. Pay close attention to which function, because this is the part that bites people.

service: api-gateway
replicas: 3
ports:
  - 8080
  - 8443

import yaml
 
with open("config.yaml") as f:
    cfg = yaml.safe_load(f)
 
print(cfg)
print("first port:", cfg["ports"][0])

safe_load did what you would expect: it turned the text into a plain dict of strings, numbers, and a list. But YAML carries a second, darker feature. The format can hold tags that tell the parser to build any Python object, including one that calls a function while it is being constructed. The general-purpose yaml.load reader honors those tags. Feed it a config file whose author is hostile and it will run their code on your machine, with your permissions, the moment you load it. This is a real RCE (Remote Code Execution, an attacker running commands on your box), and it has hit shipped software.

import sys, yaml
 
# A config that arrived from somewhere you do not fully trust.
payload = '!!python/object/apply:os.system ["echo pwned"]'
 
print("safe_load refuses the tag:", flush=True)
try:
    yaml.safe_load(payload)
except yaml.constructor.ConstructorError as e:
    print(" ", str(e).splitlines()[0], flush=True)
 
print("\nyaml.load with an unsafe Loader runs it:", flush=True)
sys.stdout.flush()
result = yaml.load(payload, Loader=yaml.UnsafeLoader)
print("os.system returned:", result, flush=True)

The payload is a single line of YAML. safe_load refuses it, because the safe reader only knows how to build strings, numbers, lists, and dicts, and it raises a ConstructorError the instant it meets a tag it does not recognize. The unsafe reader honors the tag, calls os.system("echo pwned"), which prints pwned and returns its exit code of 0. Swap that harmless echo for a line that downloads and runs a script from the internet, and you see the real shape of the problem.

One pattern runs under all three formats: the data crossing into your script came from somewhere else, so you treat it as untrusted until proven otherwise. JSON and CSV are inert. They only ever hand you strings, numbers, and containers, so the danger there is bad data, never running code. YAML can hand you a live process, so it needs the safe door every single time. Get the reading and writing solid and you have the core of every report-shuffling, alert-enriching, ticket-filing script you build next.

Related