Choosing SLIs and SLO targets: ratios, not feelings
A good SLI is a ratio in [0%, 100%] that correlates with user pain — not CPU usage or queue depth — and the right SLO target is the lowest number users tolerate, because each extra nine multiplies engineering cost by 3–10x.
A team alerts on CPU usage as their primary reliability signal. During autoscaling under healthy traffic, CPU spikes — and the pager fires. No user was ever harmed. The SLI was wrong.
What makes a good SLI
Ask yourself: if this metric looks fine, could users still be suffering? If yes, it is not a good SLI. The Google SRE workbook is firm: a good SLI is a ratio of good events to total events, in [0%, 100%], that correlates with what users feel. Four properties follow from this definition.
It must be a ratio, not a count. “Less than 100 errors per day” conflates traffic volume with reliability. At 1,000 req/day that is 10% errors; at 1,000,000 req/day it is 0.01%. The ratio form makes SLOs comparable across traffic levels and makes the error budget directly computable: budget = (1 − SLO) × total_events.
It must land in [0%, 100%]. This makes dashboards readable (fixed axis, intuitive range) and burn rate computable (error_rate / budget_rate).
It must track user pain, not machine pain. The forbidden anti-pattern is an “internal SLI”: CPU usage, queue length, GC pause duration, heap utilization. These are USE-style operational signals — they describe the machine, not the user. A server at 100% CPU may be serving every request perfectly. An SLI on CPU would fire alerts during a healthy autoscaling event while users are happy.
Signal categories by service type:
| Service type | Availability SLI | Latency SLI |
|---|---|---|
| Request-driven API | successful_requests / total_requests | requests_under_200ms / total_requests |
| Data pipeline | records_processed / records_arrived | records_within_SLA / total_records |
| Storage | data_intact / data_stored | reads_under_threshold / total_reads |
Latency SLIs use bucket counts, not percentiles
A latency SLO (“99% of requests under 200ms”) sounds like a percentile, but it is implemented as a counter. The Prometheus histogram at the SLO threshold gives you exactly this:
fast_requests = http_request_duration_seconds_bucket{le="0.2"}
latency_sli = sum(rate(fast_requests[1h])) / sum(rate(http_request_duration_seconds_count[1h]))No histogram_quantile required — and no estimation error that would corrupt the budget. This is why your RED-Duration histogram must have a bucket boundary exactly at the SLO threshold: without it, you cannot evaluate the SLO without approximating, and approximations contaminate the budget.
Choosing the SLO target
The SLO target is a business decision, not an engineering one. It answers: “what is the minimum reliability users will accept before they notice and complain?” Engineering then asks: “what is the cheapest architecture that delivers that?”
- 99% SLO, 30 days
- 7.2 hours allowed downtime
- 99.9% SLO, 30 days
- 43.2 minutes
- 99.95% SLO, 30 days
- 21.6 minutes
- 99.99% SLO, 30 days
- 4.3 minutes
- 99.999% SLO, 30 days
- ~26 seconds
- Engineering cost jump per nine
- 3–10x
The pattern:
- 99% → 99.9%: add monitoring and basic alerting
- 99.9% → 99.99%: add multi-region with automated failover
- 99.99% → 99.999%: add N+2 redundancy, chaos engineering, 24/7 on-call that wakes within minutes
The error budget arithmetic in full
Once you have the SLO, the budget becomes concrete:
error_budget = (1 − SLO) × total_events_in_window
For a 99.9% SLO over 28 days at 1 million requests per day:
- Total events = 28,000,000
- Budget = 0.001 × 28,000,000 = 28,000 failed requests
As failures accumulate: budget_remaining = budget_total − failures_so_far
Burn rate at any moment: burn_rate = current_error_rate / (1 − SLO)
At 0.1% error rate and 99.9% SLO: burn rate = 0.001 / 0.001 = 1x (sustainable). At 1.44% error rate: burn rate = 0.0144 / 0.001 = 14.4x (budget gone in 2 days).
Why 28 days, not “this calendar month”
Calendar months vary (28–31 days) and create cliff effects: a bad week at the end of February impacts the budget differently than the same bad week at the end of March. A 28-day rolling window (four whole weeks) solves both problems: it always covers the same length, and it includes complete weekday/weekend cycles, so traffic patterns normalise. Every major SLO platform — Datadog, Nobl9, Sloth, Pyrra, Google Cloud SLOs — defaults to 28 days for this reason.
▸Why this works
The rolling window is also why SLO targets should start loose and tighten quarterly, not be set tightly on day one. A conservative first SLO gives the team time to see what the real baseline error rate is, understand the traffic pattern, and instrument the counters correctly before they are held to a number that is either unmeetable or trivially easy.
Which of these is a good SLI for a request-driven API?
A 99.9% availability SLO over 28 days serves 1M requests per day. The team is 14 days in and has had 6,000 failed requests. How is the budget?
Why is a 28-day rolling window preferred over a calendar month?
- 01Why should an SLI be expressed as a ratio rather than an absolute count or a machine metric?
- 02Why does a latency SLI require a histogram bucket boundary exactly at the SLO threshold?
- 03How do you decide which SLO target to pick?
A good SLI is a ratio of good events to total events — always in [0%, 100%], always tracking what users experience, never a machine metric like CPU or queue length. Latency SLIs use histogram bucket counts at the SLO threshold, not histogram_quantile estimates, so the budget arithmetic is exact. The SLO target is a business decision: pick the lowest reliability users tolerate, because each additional nine multiplies engineering cost 3–10x. Use a 28-day rolling window to normalise traffic patterns and avoid month-boundary cliff effects. Budget = (1 − SLO) × total_events; burn rate = current_error_rate / (1 − SLO). Start loose, tighten quarterly. Now when you look at a dashboard metric and ask “is this a good SLI?” — check whether it is a ratio, whether it tracks user pain, and whether it lives in [0%, 100%].
Practice
Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.
- Low-traffic SLOs and burn-rate math from first principlessenior
- Iceberg SLIs, composite SLO math, and SLA vs SLOsenior
- Production SLO failures, self-observability, security, and the big picturesenior
- SLO and error budgets: instrument a journey end to endsenior
- SLO and error budgets: multiple-choice reviewsenior
- SLO and error budgets: PromQL and rule readingsenior
- SLO and error budgets: free-recall reviewsenior
appears again in170
- The journey of a request: seven stops from socket to responsejunior
- Accept and parse: from kernel queue to a typed requestmiddle
- Routing and middleware: choosing what runs, and in what ordermiddle
- Handler and response: from business logic to bytes on the wiremiddle
- Streaming and backpressure: when the client reads slower than you writesenior
- Timeouts and tail latency: budgets, deadlines, and the fan-out trapsenior
- Middleware and DI: the two patterns that shape every backendjunior
- Writing middleware: signatures, next(), and the three framework modelsmiddle
- Inversion of control: how dependencies reach a classmiddle
- DI scopes and lifecycles: singleton, request, transientmiddle
- DI as a testing seam: fakes, mocks, and the boundary that matterssenior
- DI containers in production: resolution graphs, circular deps, and when not tosenior
- Blocking vs non-blocking I/O: two ways to waitjunior
- The event loop: one thread, ordered phasesmiddle
- What blocks the loop: CPU work and sync callsmiddle
- Offloading CPU work: worker threads and the libuv poolmiddle
- Backpressure and bounded concurrencysenior
- Throughput under load: tail latency and saturationsenior
- Why pool: the cost of creating a connectionjunior
- Pool sizing: why bigger is not fastermiddle
- Acquisition and timeouts: the wait queue is the real latency dialmiddle
- Retry strategies: backoff, jitter, and thundering herdmiddle
- Observability, production failures, and global-scale designsenior
- Tasks, microtasks, and scheduler.yield()middle
- Timer accuracy, throttling, and idle workmiddle
- Node.js event loop: phases, nextTick, and loop lagsenior
- Rendering strategies: SSG, SSR, ISR, streaming, and hydrationjunior
- SSG, SSR, ISR, streaming, and RSC — how each worksmiddle
- Hydration cost: selective, progressive, islands, resumabilitymiddle
- Core Web Vitals: what LCP, INP, and CLS measurejunior
- LCP: four phases, one dominant costmiddle
- INP: input delay, processing, presentationmiddle
- Lab vs field: why the two disagree and how to use eachmiddle
- Metric tradeoffs, RUM attribution, and the CI+field loopsenior
- The full picture: URL to LCP to INP as a relay racejunior
- Eight layers traced: from the service worker to the second navigationmiddle
- Five canonical breaks: where production reliably diessenior
- The three-track method: reading traces and building a monitored systemsenior
- What an index is and how it speeds up queriesjunior
- The leading-column rule and composite index designmiddle
- Partial, expression, and covering indexesmiddle
- Index types: GIN, GiST, BRIN, Hash, Bloom, and HOT updatesmiddle
- Index-only scans, the Visibility Map, and INCLUDEsenior
- Production failure modes and the index audit playbooksenior
- Index design exercise: full-text search strategysenior
- EXPLAIN and execution plans: what the planner decides and whyjunior
- Scan types: Seq, Index, Bitmap, Index-Onlymiddle
- Join algorithms and the row-estimate cascademiddle
- pg_statistic, ANALYZE, and production observabilitymiddle
- Extended statistics: fixing correlated-column estimate failuressenior
- Plan cache, cost-constant tuning, and planner internalssenior
- Production failure modes and plan stabilitysenior
- Connection pools: amortising the cost of a Postgres backendjunior
- PgBouncer session, transaction, and statement modesmiddle
- Pool sizing: the (cores × 2) + spindles formula and the two-layer stackmiddle
- Pool exhaustion and idle-in-transaction: the 3 AM failure modemiddle
- Migrating to transaction mode: rollout playbook and PgBouncer 1.21 prepared statementsmiddle
- The Postgres process model and why raising max_connections degrades throughputsenior
- Pooler landscape 2026, serverless connection storms, and the full failure-mode taxonomysenior
- ADD COLUMN: instant in PG 11+ vs rewrite in older Postgresjunior
- The lock-queue failure mode: why instant DDL can freeze the databasemiddle
- Safe DDL patterns: NOT VALID, CONCURRENTLY, and unsafe-op fixesmiddle
- Migration failure taxonomy and production disciplinesenior
- Shard-key selection: hash, range, list, and directory strategiesmiddle
- Co-location and Citus: the invariant that makes sharding usablemiddle
- The hot-shard failure mode: detection, isolation, and durable policymiddle
- Online resharding, 2PC, and the operational cost of shardingsenior
- The seven acts: from CREATE TABLE to Citusjunior
- Acts 1–3 in depth: schema, indexes, and planner statisticsmiddle
- Acts 4–6 in depth: MVCC bloat, connection pooling, and safe migrationsmiddle
- Act 7 in depth: sharding, co-location, and the seven-tier tradeoff cascademiddle
- Observability, anti-patterns, and production triagesenior
- Generational GC and the Scavengermiddle
- Major GC: mark, sweep, compactsenior
- Write barriers: the price of incremental and generational GCsenior
- Bits on the wirejunior
- Latency mathmiddle
- Bufferbloat and congestionsenior
- The physical frontiersenior
- Sequence numbers and connection statemiddle
- Flow control and congestion controlmiddle
- BBR, production observability, and beyond TCPsenior
- CDN: putting content next doorjunior
- Anycast and GeoDNS: routing to the nearest edgemiddle
- Tiered cache and Cache-Controlmiddle
- Vary header and cache keysmiddle
- Stale-while-revalidate and cache stampedesenior
- Edge workers and edge-side compositionsenior
- CDN operations and observabilitysenior
- WebSocket: the HTTP upgrade handshakejunior
- WebSocket vs SSE vs long-polling: choosing the right transportmiddle
- WebSocket backpressure: when clients can''''t keep upmiddle
- Reconnection: jittered backoff, thundering herd, message resumptionsenior
- WebSocket at scale: HTTP/2 multiplexing, permessage-deflate, C10Msenior
- WebSocket in production: proxies, security, and distributed architecturesenior
- What reverse proxies dojunior
- Balancing algorithms: round-robin to power-of-two-choicesmiddle
- L4 vs L7 load balancing and client-IP preservationmiddle
- Health checks, connection draining, and slow startmiddle
- Retry storms, circuit breakers, and load sheddingsenior
- Resilient LB architecture: anycast, zone-aware routing, and observabilitysenior
- Why QUIC and not TCP+TLSjunior
- QUIC streams and head-of-line blockingjunior
- Integrated handshake and 1-RTTmiddle
- Connection IDs and network migrationmiddle
- Loss detection and congestion controlmiddle
- 0-RTT resumption and packet encryptionsenior
- Deployment tradeoffs and CPU costsenior
- DDoS: what it is and why it worksjunior
- Amplification attacks and state exhaustionmiddle
- Rate limiting: algorithms and architecturemiddle
- WAFs, firewalls, mTLS, and HSTSmiddle
- DNS cache poisoning and BGP hijackingsenior
- Defense-in-depth architecture and attack economicssenior
- The twelve layers: one URL, seven actorsjunior
- DNS, TCP, TLS in sequence: where the milliseconds gomiddle
- Critical render path and Core Web Vitalsmiddle
- Proxy intercepts and security gates: rate limiters, WAF, mTLSmiddle
- Alternate paths: QUIC 0-RTT, WebSocket upgrade, connection migrationmiddle
- Observability: distributed traces, USE/RED, and samplingsenior
- Resilience: cascading retries, circuit breakers, and error budgetssenior
- Why profile first: measure where time actually goesjunior
- Amdahl''''s law and self-time: the ceiling on every speedup you can shipmiddle
- The measurement loop: microbench, macrobench, prod profile, observer effectmiddle
- Reading flame graphs: shapes, per-language profilers, and the 60-second scanmiddle
- Statistical baselines: why one run is not a measurementmiddle
- Profiler history and microbenchmark pitfalls: Knuth to GWPsenior
- Hardware counters, cold-start profiles, and profile securitysenior
- Continuous profiling at scale: costs, CI gates, trace correlation, and anti-patternssenior
- What makes a hot path: symptom vs causejunior
- Five shapes of hotspot: CPU, alloc, cache, lock, syscallmiddle
- Reading parent and child chains: where to apply the fixmiddle
- JIT deopt, the fix-and-verify loop, and PR-time profilingmiddle
- Hardware counters and Intel TMA: sub-category diagnosissenior
- False sharing and native-bridge hot pathssenior
- Hot paths in production: security, tail latency, and tooling lineagesenior
- Memory hierarchy: why the same O(N) loop can be 17x slowerjunior
- Row-major vs column-major: access order and the 9x gapjunior
- Branch prediction and branchless codemiddle
- Hardware prefetcher, TLB, and memory-level parallelismsenior
- GC basics: what the runtime taxes you forjunior
- GC algorithms: generational, concurrent, and per-runtimemiddle
- GC tradeoffs: pause, throughput, heap — and object poolingmiddle
- GC tuning: pacing, heap shape, and allocation observabilitymiddle
- GC internals: tri-color invariant, write barriers, and per-runtime deep-divessenior
- GC in production: observability, security, edge cases, and fleet governancesenior
- N+1: one logical operation, many round-tripsjunior
- Fix families: JOIN, IN, preload, and DataLoadermiddle
- Detecting N+1: query logs, APM traces, and CI gatesmiddle
- DataLoader: batching across resolver treesmiddle
- Cross-protocol N+1: HTTP fan-out and Redis MGETmiddle
- N+1 at scale: pool exhaustion, plan changes, and denormalisationsenior
- Batching: amortize fixed cost per operationjunior
- The batching window: size and wait timemiddle
- Batching in Kafka and Postgresmiddle
- io_uring and observability of batchingmiddle
- From Nagle to io_uring: evolution of batchingmiddle
- Backpressure, failure isolation, and batch security in productionsenior
- What a bundle actually costs: download, parse, compile, executejunior
- Core Web Vitals: LCP, INP, and CLSmiddle
- Code splitting: route-level, component-level, vendor splittingmiddle
- Tree shaking and compression: removing what you don''''t usemiddle
- Third-party scripts: the silent budget killermiddle
- CI enforcement and RUM: making budgets stickmiddle
- V8 JIT pipeline, HTTP priorities, and bundle securitysenior
- The performance loop: discipline, not a projectjunior
- Classify and fix: matching bottleneck families to remediesmiddle
- Observability stack and CI gates: catching regressions before they shipmiddle
- Incident to enforcement: SLO burn to verified fix in 35 minutesmiddle
- Culture, economics, and org-scale performancesenior
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.