Observability
SLO and error budgets: multiple-choice review
Crux Multiple-choice synthesis across the SLO unit — budget arithmetic, burn-rate derivation, MWMBR AND logic, composite ceilings, and the organisational failure modes.
Your altitude — climbing toward senior
ZeroJuniorMiddleSenior
You are at senior altitude — in orbitSix questions that cut across the whole unit. Each one is a decision you make in a real reliability review — not a definition to recite, but a number to compute and a tradeoff to defend.
Goal
Confirm you can connect SLI selection, budget arithmetic, burn-rate derivation, MWMBR alerting, composite-journey math, and the organisational layer — the synthesis the eight lessons built toward.
Quiz
Completed
A service runs a 99.9% availability SLO over a 28-day window at 1M requests/day. Fourteen days in it has served 14M requests with 21,000 failures. How healthy is the budget?
Heads-up Position in the window matters. At day 14 the expected spend is half the budget (14,000). 21,000 spent is 75% of the budget with 50% of the window remaining — a ~1.5x burn that exhausts the budget early.
Heads-up Most policies freeze at zero remaining, not at 75% spent. The team should slow risky deploys and watch the burn, but the freeze trigger has not been reached yet.
Heads-up The error budget derives from the SLO and the traffic, not the SLA. Budget = (1 − SLO) × total_events; the SLA is a separate, looser customer commitment.
Quiz
Completed
An on-call team copies a 1h+5m page rule from a 99.9% SLO service — expr (1 − ratio_rate1h) > (14.4 × 0.001) — onto a service whose SLO is 99.5%, leaving the expression unchanged. What breaks?
Heads-up The burn multiplier (14.4x) is the same, but the error-rate threshold it maps to is burn × (1 − SLO). The copied 0.001 encodes a 99.9% budget; for 99.5% it must become 0.005, or the alert fires ~5x too eagerly.
Heads-up It fires more, not less. The leftover 0.001 sets the firing point at a 1.44% error rate, which a 99.5%-SLO service exceeds routinely — the opposite of never firing.
Heads-up Both windows compare against the same hard-coded budget rate. This is exactly why hand-rolled PromQL is error-prone and platforms like Sloth generate it from the declared target.
Quiz
Completed
An MWMBR page rule fires when the 1h burn AND the 5m burn both exceed 14.4x. During an incident the 1h burn is 18x but the 5m burn has dropped to 9x. Should it page, and what does each window tell you?
Heads-up MWMBR deliberately requires BOTH windows. The short window is the 'is it still happening now?' gate; with it below threshold the AND is false and the page suppresses, avoiding a wake-up for an already-recovering incident.
Heads-up 9x is high in absolute terms but below the 14.4x page threshold. The AND needs both windows over the line; the falling short window is the recovery signal that keeps the page quiet.
Heads-up The AND between the two windows lives in the PromQL expression itself. 'for' is a separate debounce mechanism and does not implement the dual-window logic.
Quiz
Completed
A checkout journey traverses API gateway, auth, inventory, payment, and database, each holding an independent 99.9% SLO, all green. The user-facing checkout success rate sits at ~99.5%. What is the correct reading and fix?
Heads-up Per-service SLOs compose as a product, not an average or a max. Five independent 99.9% services give 0.999^5 ≈ 99.5%; the dashboards are all correct, they just measure the wrong layer for the user journey.
Heads-up That works mathematically (0.9999^5 ≈ 99.95%) but is the expensive lever — each extra nine costs 3-10x. The cheaper fix is a journey-level SLI plus targeting the Pareto-dominant contributors, or idempotent retries / hedging on the worst hop.
Heads-up Window length changes detection speed, not the composite ceiling. The journey is structurally capped at ~99.5% regardless of window; only journey-level measurement and architectural levers raise it.
Quiz
Completed
A checkout availability SLI counts any 2xx as 'good'. It is steadily green, yet customers report duplicate charges and 30-second 'successful' checkouts. Why does the SLO miss this, and what is the senior fix?
Heads-up No target catches a failure the SLI never observes. A 99.999% SLO with a 2xx-only SLI still scores duplicate charges as 'good'. The defect is the indicator, not the objective.
Heads-up Thresholds are irrelevant when the SLI does not count the failure. If correctness errors are invisible to the indicator, no alert tuning will ever surface them.
Heads-up Composition is a different problem. Here a single SLI fails to track correctness and latency; the fix is multiple user-outcome SLIs joined worst-of, not re-slicing per service.
Quiz
Completed
Six months after rolling SLOs to 80 services with correctly generated MWMBR alerts and a signed error budget policy, half the teams ignore the pages and budgets go negative without freezes. What is the most likely root cause?
Heads-up If the SLI is wrong, no threshold fixes it — a perfectly calibrated burn rate on a meaningless indicator still pages for non-issues, and raising thresholds delays real detection. The fault is the SLI definition and policy enforcement.
Heads-up Better tooling does not fix an organisational bottleneck. Teams that switch tools without fixing the SLI and the policy get the same dashboards nobody acts on. Fix the contract first, then the tools.
Heads-up Shortening the window changes freeze cadence, not whether alerts are relevant or whether the policy has teeth. Negative budgets with no freeze is an enforcement gap, not a window-length problem.
Recap
The unit’s through-line is one chain of arithmetic made organisational: SLI = good/total (and it must track user pain, not 2xx), SLO is the target, error budget = (1 − SLO) × events, and burn rate normalises the spend. Alerting derives from burn = (budget_fraction × period) / window, fired with a long-AND-short window so it is both noise-resistant and fast to reset; remember to rebase the budget rate when the SLO target changes. Journeys multiply (0.999^5 ≈ 99.5%), iceberg SLIs hide correctness behind 2xx, SLO must sit tighter than SLA, and the framework only delivers value when a signed, enforced policy turns the number into a decision.