Multiple-choice synthesis across the availability unit: SLI/SLO/SLA and the nines, error budgets, redundancy sizing and correlated failure, and the failover/split-brain/retry-storm traps that turn recovery into outage.
SDSenior◷ 13 min
Level
FoundationsJuniorMiddleSenior
Six questions that cut across the whole unit. Each is a judgement you make when you write an SLO, size redundancy, review a failover design, or read a postmortem — not a definition to recite, but the reasoning behind where to set a target, what counts as redundant, and why a failover turned into an outage.
Confirm you can place an SLO under an SLA, do the nines and availability-composition arithmetic, spend an error budget, size redundancy to N+1 against peak, spot a correlated failure behind “redundant” copies, and recognize split-brain and the retry storm.
Quiz
Completed
A customer SLA promises 99.9% monthly availability with credits for misses. Where do you set the internal SLO, and why?
Heads-up If the SLO equals the SLA, you only learn you're in trouble once the contract is already breached and money is owed. The SLO must sit tighter to function as an early warning.
Heads-up Backwards: a looser SLO would routinely let you breach the SLA while the internal dashboard still looks green. The SLO has to trip BEFORE the contract, so it's set tighter.
Heads-up The SLA has no early warning; the SLO is the internal target whose breach alerts you in time to act. Without it you discover the miss at payout time.
Quiz
Completed
Your SLO is 99.99% successful requests over 28 days. Roughly how much downtime/failure does that allow per month, and what does the leftover represent?
Heads-up 43 minutes is three nines (99.9%), not four. Four nines (99.99%) is ~4.3 min/month. And the leftover (100% − SLO) is the error budget, not the SLA — the SLA is the external contract.
Heads-up 26 seconds is five nines (99.999%). Four nines is ~4.3 min/month, and the 0.01% allowed to fail IS a spendable error budget — that's the whole point of expressing the SLO this way.
Heads-up 99.99% explicitly allows 0.01% to fail — ~4.3 min/month. Treating it as 'never fail' discards the error budget, the resource that lets you ship features without breaching the objective.
Quiz
Completed
A request path is four independent serial components, each at 99.9% availability. What's the path's availability, and the design lesson?
Heads-up Availability composes by multiplication, not by copying one component's number. Four serial 99.9% links give 0.999⁴ ≈ 99.6%, lower than any single link.
Heads-up Adding serial dependencies LOWERS availability — each is another thing that can fail the request. 0.999⁴ ≈ 99.6%, not 99.99%.
Heads-up Serial-availability composition needs only each link's availability: multiply them, 0.999⁴ ≈ 99.6%. Latency is a separate axis.
Quiz
Completed
A service needs 4 servers to carry peak and runs exactly 4, calling itself redundant. One dies at peak. What's the verdict?
Heads-up Redistribution doesn't create capacity. Peak needs 4; with 3 the survivors are overloaded and tip toward collapse. You needed N+1 (a 5th) to tolerate one loss.
Heads-up Redundancy means surviving a loss while still meeting peak, which requires spare capacity (N+1). Exactly N puts you under capacity on the first failure.
Heads-up Capacity is exactly what's hit — 25% of it gone at peak — and the survivors can't absorb it. The latency spike is the symptom of the resulting queueing collapse.
Quiz
Completed
Two database replicas are run 'for redundancy,' but both sit in the same availability zone. The AZ loses power. What does this expose?
Heads-up Two copies in one failure domain aren't redundant; a single event kills both. Independence is the whole point, and same-AZ placement correlates the failures.
Heads-up That multiplication assumes INDEPENDENT failures. A shared AZ correlates them, so real availability is governed by the AZ, not the product of the replicas.
Heads-up More copies inside the same shared dependency don't help — the AZ event still takes all of them. Spread across independent failure domains instead.
Quiz
Completed
A network partition isolates a primary from its standby; the standby promotes itself while the primary is still alive and taking writes. What is this and what prevents it?
Heads-up It's split-brain, and there's no automatic reconcile — two conflicting write histories. The failover misfired because it couldn't tell 'dead' from 'unreachable.' Quorum/fencing prevents it.
Heads-up A longer timeout trades risks — fewer false positives but slower response to a REAL death — and never resolves the dead-vs-unreachable ambiguity. The durable fix is quorum or fencing.
Heads-up It's a correctness problem, not capacity: two nodes both believe they're primary. A bigger box doesn't help; ensuring only one can win (quorum/fencing) does.
Recall before you leave
01
Why does 'two replicas, both in one AZ' fail the redundancy test?
02
What single design move prevents a split-brain during a network partition?
Recap
The through-line of the unit is that availability is reasoning you do before the incident. You set the SLO under the SLA so an internal breach warns you before you owe credits; you do the nines arithmetic (99.9% ≈ 43 min/month, 99.99% ≈ 4.3 min, 99.999% ≈ 26 s) and remember availability multiplies across serial dependencies, so a four-link 99.9% path is only ~99.6%. The error budget (100% − SLO) turns reliability into a spendable resource. You size redundancy to N+1 against peak, not average, and you place the copies in independent failure domains because two replicas sharing an AZ is correlated failure, not redundancy. And you design failover for a wrong detection: quorum or fencing against split-brain, bounded timeouts and capped retries with backoff-and-jitter against the retry storm, so recovery doesn’t become the next outage — with availability ≈ MTBF/(MTBF+MTTR) reminding you that fast, rehearsed recovery (MTTR) usually beats chasing a higher MTBF.
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.