Detection as code

Version-controlled, tested, CI/CD-deployed detections.

Fifteen years ago a server got configured by hand. Somebody logged in over SSH (secure shell, the encrypted remote terminal), typed a dozen changes, and hoped they would still remember them next quarter. Infrastructure as code ended that habit by turning those changes into reviewed, versioned files anyone could reproduce. Detection as code does the same thing to your SIEM (security information and event management platform, the system that swallows your logs and raises alerts). A detection is logic that reads telemetry, meaning logs, events and network records, and raises an alert when something matches. That makes it software in every sense that counts. Yet most security operations centres still hand-type their rules into a web console. Detection as code moves every rule into a git repository, pushes every change through peer review and automated tests, and lets CI/CD (continuous integration and continuous delivery, the same pipeline machinery that ships application code) deploy validated rules to the SIEM.

Why go to the trouble? Because a rule that lives only in a console fails in four predictable ways. Nobody reviewed it, so a typo in an exclusion quietly switches it off. It has no tests, so the day a log source renames a field, the rule stops firing and no one notices. It has no history, so six months later nobody can say why it ignores an entire subnet. And it exists inside one vendor's database, so a SIEM migration wipes out your detection programme. Version control fixes all four at once. git blame answers who changed it and why, git revert turns a bad rule into a one-line rollback, and the repository itself is a vendor-neutral record that outlives any platform.

One rule, one file: what the repo looks like

The unit of work is a single rule in a single YAML file (YAML is the indentation-based text format most config is written in today), and the language is Sigma, the vendor-neutral signature format the next lesson pulls apart line by line. What matters right now is the label every file carries, the way a prescription bottle carries dose, expiry and prescriber. Each rule declares a stable id (a UUID, a universally unique identifier that never changes), a status of experimental, test or stable, a severity level, MITRE ATT&CK tags (ATT&CK is MITRE's public catalogue of attacker tactics and techniques), a pointer to the intel or the incident that motivated the rule, and notes on the false positives you already know about. The test samples sit right beside it: one event the rule *must* match, one it *must not*. A reviewer opens one diff and sees the logic and the evidence together.

$ tree -a -I '.git' -L 3 detections/
detections/
├── .github
│   └── workflows
│       └── detections.yml
├── pipelines
│   └── custom_fieldmap.yml    # org-specific field mappings
├── rules
│   ├── credential_access
│   │   └── aws_console_login_no_mfa.yml
│   └── discovery
│       └── proc_creation_whoami_priv.yml
└── tests
    └── proc_creation_whoami_priv
        ├── negative.json      # event that MUST NOT fire
        └── positive.json      # event that MUST fire
 
8 directories, 6 files
 
$ git log --oneline -3 -- rules/discovery/proc_creation_whoami_priv.yml
9f3c2ab tune: exclude SCCM health-check parent (FP ticket SOC-1841)
41d0e77 test: add negative sample for Nessus scanner service account
c7b19e4 feat: detect whoami /priv after initial access (HUNT-207)

Read that git log output the way an auditor would. The rule exists because a hunt, HUNT-207, found the behavior in the first place. A scanner false positive got handled with an explicit negative test instead of a silent exclusion. The tuning change traces back to a ticket someone can still open and read. The UUID in the file carries the same weight. It is the stable identity your deploy step uses to *upsert* the rule into the SIEM (update it if it is already there, insert it if it is not), so re-running a deployment refreshes rules in place rather than breeding duplicates.

sigma-cli: write the rule once, compile it everywhere

A folder of YAML files catches nothing on its own. It has to become queries your SIEM can actually execute, and that translation is a build step, no different in spirit from compiling source code into a program. The tool is sigma-cli, the command-line front end to pySigma, the Python library doing the real work. The compile runs in three stages. The *parser* reads the YAML into a rule object and checks it against the Sigma schema. A *processing pipeline* then reshapes that rule for your environment, mostly field mapping (Sysmon's Image becomes process.executable.caseless, the case-insensitive field name in ECS, the Elastic Common Schema) plus any index or source scoping you need. Finally a *backend* walks the rule's condition tree and writes it out in the target query language: SPL (Search Processing Language) for Splunk, Lucene or ES|QL for Elastic, KQL (Kusto Query Language) for Microsoft Sentinel.

$ pipx install sigma-cli
$ sigma plugin install splunk
$ sigma plugin install elasticsearch
$ sigma plugin install sysmon   # pipeline plugin: generic categories -> Sysmon event IDs
 
# gate 1: parse and validate every rule in the repo
$ sigma check rules/
# => Found 0 errors, 0 condition errors and 2 issues.
 
# compile the same rule for Splunk...
$ sigma convert -t splunk -p sysmon -p splunk_windows rules/discovery/proc_creation_whoami_priv.yml
source="WinEventLog:Microsoft-Windows-Sysmon/Operational" EventCode=1
  Image="*\\whoami.exe" CommandLine="*/priv*"
 
# ...and for Elastic, with ECS field names
$ sigma convert -t lucene -p ecs_windows rules/discovery/proc_creation_whoami_priv.yml
process.executable.caseless:*\\whoami.exe AND process.command_line:*\/priv*

Compare the two outputs. The logic did not move. Only the dialect and the field names did. Those -p pipeline flags are where correctness actually lives, and they apply in order. sysmon resolves the generic *process_creation* category into Sysmon Event ID 1. Then splunk_windows renames EventID to Splunk's EventCode and scopes the search to the Sysmon event log source, which is the shape your logs have when they arrive through the Splunk Add-on for Windows. ecs_windows does something different. It maps fields for data that Elastic Agent has already normalised to ECS. Choose the wrong pipeline and you get a flawless query pointed at fields your data never fills. Shops running more than one SIEM (common after a merger, guaranteed during a migration) compile one source tree out to every backend they own.

The pipeline: every merge is a deployment

The pipeline is what turns good intentions into discipline. Every pull request runs three gates. *Validate*: sigma check rejects schema errors and missing metadata. *Test*: replay the positive and negative samples against the rule, and the mechanics of that replay get a whole lesson of their own, Testing detections. *Compile*: prove every rule converts cleanly for every backend you target. Only a merge into main reaches the fourth stage, *deploy*. The pull request becomes the detection review itself. Your reviewer sees the logic diff, the test evidence and the threat context in one place, which is a far higher bar than a console edit nobody witnessed.

name: detections
on:
  pull_request:
  push: { branches: [main] }
 
jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-python@v6
        with: { python-version: "3.12" }
      - run: pip install sigma-cli && sigma plugin install elasticsearch
      - run: sigma check rules/                # gate 1: schema + lint
      - run: python scripts/replay_tests.py    # gate 2: positive/negative samples
      - run: mkdir -p compiled && sigma convert -t lucene -p ecs_windows rules/ -o compiled/lucene.txt
 
  deploy:
    needs: validate
    if: github.ref == 'refs/heads/main'        # PRs validate; only main deploys
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - run: ./scripts/deploy.sh               # compiles, then upserts by rule UUID
        env:
          KIBANA_URL: ${{ vars.KIBANA_URL }}
          ELASTIC_API_KEY: ${{ secrets.ELASTIC_API_KEY }}

The deploy script is a loop of idempotent API calls (API is an application programming interface, the machine-to-machine door into the SIEM), one call per rule, keyed on the UUID from the YAML. POST creates a rule the SIEM has never seen. PUT replaces one it already has. Idempotent means running the loop ten times leaves you exactly where running it once did, the way setting a thermostat to 20 differs from nudging it up a degree at a time. The example below talks to Elastic Security's detection engine API. Splunk (/services/saved/searches), Sentinel (Azure REST or Terraform's azurerm_sentinel_alert_rule_scheduled) and Google SecOps (Chronicle) all expose an equivalent. Credentials come from the CI secret store, never from the repo.

$ curl -s -X PUT "$KIBANA_URL/api/detection_engine/rules" \
    -H "Authorization: ApiKey $ELASTIC_API_KEY" \
    -H "kbn-xsrf: true" -H "Content-Type: application/json" \
    -d @compiled/proc_creation_whoami_priv.json \
  | jq '{rule_id, name, enabled, severity, interval, version}'
{
  "rule_id": "8f2c5b1e-71a4-4c1b-9d6e-02a4f0d9b0aa",
  "name": "Whoami Privilege Discovery After Initial Access",
  "enabled": true,
  "severity": "medium",
  "interval": "5m",
  "version": 4
}

That "version": 4 in the response is the SIEM telling you it updated a rule in place. Fourth revision of the same identity, matching four commits in git. Retiring a rule is a pull request that flips enabled to false or deletes the file, and either way the change is reviewed and reversible. After a platform migration you redeploy the whole programme from the repo in a single pipeline run.

Where the model breaks, and how to run it at scale

Be honest about what does not fit. Not every detection compiles out of Sigma. Support for multi-event correlation (Sigma v2 correlation rules) is patchy across backends, and machine-learning jobs or UEBA baselines (user and entity behaviour analytics, the models that learn what normal looks like for an account or a host) are native SIEM constructs with no portable representation at all. Keep them in the repo anyway, exported as native JSON or Terraform. Detection as code is a *discipline*, not a file format. Generated queries can also run slower than a hand-tuned native one, so on a SIEM billed by consumption you measure before you assume the compiler produced something efficient. And incidents need a hotfix lane. Allow the emergency console edit, require a back-port pull request within 24 hours, and run a nightly job that diffs live SIEM state against the repo so drift gets detected rather than discovered.

The economics pay you back. Every scheduled rule burns SIEM compute, and 800 rules on 5-minute intervals is a line item somebody in finance can see. In a code-reviewed repo, changing an interval is a visible diff a human has to approve, not a quiet console toggle at 2am. At scale the repo becomes the substrate for everything downstream. Fire-rate metrics and false-positive tickets attach to commits, coverage maps attach to ATT&CK tags, and the improvement loop you build in later lessons finally has a concrete artifact to act on.

Everything in this pipeline, the review, the tests, the compile, the deploy, operates on one artifact: the Sigma rule file. Its logsource declaration, its detection blocks, its field modifiers and its condition logic are what make a rule portable, precise and reviewable, and getting them right is a craft in itself. That format is where the next lesson goes.

Try this

Run this against a lab clone of your detections repo, not against production CI secrets. You are proving two things: that sigma-cli validates the rule, and that the same rule compiles for two different backends. Then read both queries side by side and notice what stayed the same.

$ pipx install sigma-cli
$ sigma plugin install splunk elasticsearch sysmon
$ sigma check rules/discovery/proc_creation_whoami_priv.yml
# => Found 0 errors, 0 condition errors and 0 issues.
 
$ sigma convert -t splunk -p sysmon -p splunk_windows \
    rules/discovery/proc_creation_whoami_priv.yml
source="WinEventLog:Microsoft-Windows-Sysmon/Operational" EventCode=1
  Image="*\\whoami.exe" CommandLine="*/priv*"
 
$ sigma convert -t lucene -p ecs_windows \
    rules/discovery/proc_creation_whoami_priv.yml
process.executable.caseless:*\\whoami.exe AND process.command_line:*\/priv*
 
# Same logic, two dialects — that is the whole point of detection as code.

Takeaway

Remember that a detection is software. Keep it in git, review it, test that it fires and that it stays quiet on the negative sample, deploy it by UUID, and watch your canaries so a green pipeline cannot hide a blind rule.

Next step: write one Sigma rule with its positive and negative samples sitting beside it, then wire sigma check and the replay script into the pull request pipeline before you touch the SIEM console again.

Related