SIEM & correlation

Ingest, correlate weak signals, manage cost.

The SIEM is where telemetry becomes detection: the platform that ingests the pipeline’s output, runs detections, correlates events into incidents, and retains data for investigation. Understanding what it does — and its cost/noise tradeoffs — is central to detection engineering.

Ingest, correlate, retain

A SIEM (Splunk, Elastic, Microsoft Sentinel, Chronicle/Google SecOps) does three jobs. It ingests and indexes normalized events for fast search. It runs detections — scheduled and near-real-time — and correlates across sources so that separate weak signals combine into one strong incident: a failed-then-successful login, plus a new IAM grant, plus data access becomes a single high-confidence alert rather than three isolated ones. And it retains data so you can investigate and hunt over history. Correlation is the SIEM’s superpower — it sees relationships across time and source that a single-log rule cannot.

# Three individually-low-signal events, correlated into one incident:
#
#   1. IAM: console login without MFA        (suspicious)
#   2. IAM: new AccessKey created for a user  (suspicious)
#   3. S3:  unusual bulk GetObject             (suspicious)
#     within 10 minutes, same principal
#   ⇒ CORRELATED: likely account compromise + exfil  → high-severity incident
#
# A SIEM correlation rule joins on the principal + time window across sources.

Cost, noise, and SOAR

Ingesting everything into a SIEM is neither free nor wise: cost scales with volume, and noise scales with poor rules. Tier your data (hot for high-value security logs, cheap archive for the rest), and invest in alert quality so the SIEM surfaces incidents, not a firehose. Pair it with SOAR (Security Orchestration, Automation, and Response) to turn alerts into action: playbooks that isolate a host, disable a user, enrich a case, or open a ticket automatically, closing the gap between detection and response. Detection as code feeds the SIEM its rules; SOAR drives what happens when they fire. Together they are the operational core of the program.