Detection engineering
A rule clicked into a SIEM console has no history, no test, no reviewer. Detection engineering treats detections as code — Sigma rules in Git, tested against sample logs, reviewed in a PR — so tuning becomes an auditable diff, not a click no one remembers.
3 a.m., the on-call analyst is drowning. A detection named “Suspicious PowerShell” has fired 4,000 times overnight — every one of them a backup job that someone added to a server eight months ago. The analyst does what every exhausted analyst does: opens the SIEM console, finds the rule, adds an exception for that host, clicks save. The alerts stop. Nobody writes it down. Six weeks later a different engineer, staring at the same noisy rule, deletes the condition entirely to make it quiet — and silently blows a hole through which the exact technique the rule was built to catch now walks unseen. There is no commit, no diff, no review, no test that would have caught it. The rule was never code. It was a setting in a box, and settings in a box have no memory.
By the end of this lesson you’ll know how to write, test, and version detections as code — Sigma rules in Git, validated against sample logs, reviewed in a pull request — so tuning a noisy rule is an auditable engineering change instead of a forgotten click.
What detection engineering actually is
You already know that SIEM rules turn logs into alerts. Detection engineering is the discipline that asks a harder question: how do you build, change, and trust those rules over years, across a team, without the rule set rotting into noise? The answer the field converged on is detection-as-code — treat every detection the way you treat application code. It lives in a Git repository, not a vendor console. It has an author, a commit message, and a reviewer. It ships through CI. And critically, it has tests: sample log events that should fire it (the true positives) and sample events that should not (the negatives), run on every change.
The console-clicking model fails for the same reasons editing production by hand in a database GUI fails. There’s no history, so you can’t answer “who changed this rule and why?” There’s no review, so a tired analyst’s 3 a.m. exception ships with zero second pair of eyes. There’s no test, so a one-character edit to a regex can quietly stop matching the thing it was built to catch and nobody finds out until the breach. Detection-as-code closes all three gaps by borrowing the muscles you already have from software delivery: version control, code review, and automated testing.
Sigma: the rule format that travels
The mechanism that makes this portable is Sigma — a generic, YAML-based signature format for log-based detections, the “write once, target many” layer for the SIEM world. You describe the logic of a detection in Sigma’s vendor-neutral schema, then a converter (the reference tool is sigma-cli with its pySigma backends) compiles that one rule into the query language of whatever backend you run: Splunk SPL, Elastic’s query DSL, a Microsoft Sentinel KQL query, and so on. Write the logic once; let the toolchain emit the dialect.
A Sigma rule’s heart is two fields. detection.selection lists the field/value conditions that must match, and condition combines selections with boolean logic. Here is a real-shaped rule for the very alert that flooded our analyst:
| Sigma field | Example value | What it does for you |
|---|---|---|
title | Encoded PowerShell command line | Human name — what shows up in the alert |
logsource | product: windows, category: process_creation | Which log stream the rule runs against |
detection.selection | Image endswith \powershell.exe; CommandLine contains ‘-enc’ | The match conditions |
detection.filter | CommandLine contains ‘BackupAgent\run.ps1’ | The tuning carve-out — in the rule, in Git |
condition | selection and not filter | Boolean glue — the actual logic |
tags | attack.execution, attack.t1059.001 | ATT&CK mapping — feeds your coverage grid |
Notice the tuning lives in filter, expressed in the condition as selection and not filter. That is the same exception our 3 a.m. analyst made by hand — but now it is a line in a YAML file, in a commit, with a message saying why the backup agent is excluded, reviewable by anyone who later wonders whether that carve-out is too broad. The fix didn’t move into a hidden console setting; it moved into the diff.
The lifecycle: a detection is software
The reason this matters is that detections are never “done.” An adversary changes a flag, a vendor renames a log field, an application starts emitting a benign pattern that looks malicious — and the rule must change. Detection-as-code gives that change a pipeline.
The test step is the load-bearing one and the part teams skip first. Every Sigma rule should ship with fixtures: at least one log event that represents the real attack technique (assert it fires) and a set of known-benign events that look superficially similar (assert they stay quiet). When the 3 a.m. analyst adds filter: BackupAgent\run.ps1, CI re-runs the malicious fixture and proves the rule still catches the encoded-PowerShell attack even with the new carve-out. The six-weeks-later engineer who tries to delete the whole condition watches the true-positive test go red in the pull request. The test is what converts “make it quiet” from a gamble into a safe operation.
▸Why this works
Why is a false positive treated as a first-class bug and not just an annoyance? Because alert fatigue is itself an attack surface. An analyst facing 4,000 false alarms a night doesn’t carefully triage the 4,001st — they reflexively dismiss the queue, and that’s exactly when the real one slips past. The infamous 2013 Target breach is the canonical case: the malware was flagged, but the alert drowned in noise and went unactioned. So a detection’s false-positive rate isn’t a cosmetic metric; it’s the difference between a control that works and one that has trained its operators to ignore it. Tuning is not optional polish — it’s keeping the control alive.
The signal you are actually tuning
Every detection change is a trade between two error types, and saying which one you’re optimizing is the whole skill. A true positive is the rule firing on real malicious activity — the thing you want. A false positive is the rule firing on benign activity — the backup job. Tighten the logic to kill false positives and you risk creating false negatives: real attacks the now-narrower rule no longer catches. Loosen it to catch more variants and your false positives climb until the analyst tunes you out. There is no setting that makes both zero; detection engineering is the practice of choosing the point on that curve deliberately, per technique, and recording the choice where the next engineer can see it.
A detection 'Encoded PowerShell' fires 4,000 times a night, all from one backup host running a known script. You need it quiet without going blind to the attack. Pick the senior move.
Why does detection-as-code keep the tuning exception inside the Sigma rule's `filter` rather than as a suppression in the SIEM console?
An engineer tightens a rule to silence false positives, and it works — alerts drop to near zero. What's the senior's first worry?
Order the detection-as-code lifecycle for a new rule, from authoring to production tuning:
- 1 Write the Sigma rule and tag it with its ATT&CK technique
- 2 Commit to Git with an author and a rationale message
- 3 CI converts the rule and runs it against TP and negative fixtures
- 4 A peer reviews and approves the pull request
- 5 Deploy the converted query to the SIEM; tune later FPs back through the same loop
- 01Explain detection-as-code and why it beats clicking rules into a SIEM console.
- 02What is Sigma, what are its key fields, and how does the false-positive / true-positive trade-off shape how you tune a rule?
Detection engineering is the discipline of building, changing, and trusting detections over years without the rule set rotting into noise — and its core practice is detection-as-code: rules live in Git, not a SIEM console, with an author, a commit message, a peer review, CI, and tests. Sigma is the vendor-neutral YAML format that makes this portable: write the logic once (logsource, selection, optional filter, condition, ATT&CK tags) and a converter emits Splunk, Elastic, or Sentinel queries. Tuning a false positive becomes a narrowly scoped filter expressed as selection and not filter, committed with a rationale and re-validated by CI against a true-positive fixture — so killing noise can never silently create a false negative. The whole thing turns on treating a false positive as a first-class bug, because alert fatigue is an attack surface: a rule that floods its operators has trained them to ignore the one alert that mattered. So the next time you face a noisy detection, your reflex is no longer “click an exception in the box” — it’s “scope a filter, write the rationale, and prove with a test that the attack still fires.”
Practice
Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.
Apply this
Put this lesson to work on a real build.