Redundancy & single points of failure

What a single point of failure actually is

A single point of failure (SPOF) is any component whose individual failure takes down the entire system. It’s the load-bearing wall: knock it out and everything above it falls. The discipline of building for availability is, at its core, the discipline of hunting down SPOFs and eliminating them — one at a time, because there is usually more than one and they hide.

The way you find them is a thought experiment you run over every box and line in your architecture diagram: “If this one thing dies right now, does the system keep serving?” The database in the hook failed that test — kill it and everything stops. So does the single load balancer in front of a “redundant” fleet, the one NAT gateway all egress traffic flows through, the single DNS provider, the one CI server that holds the only copy of your deploy keys, the lone on-call engineer who knows how the legacy billing job works. SPOFs aren’t only machines; they’re any singular dependency — including a person or a vendor.

Redundancy: N, N+1, N+2

You eliminate a SPOF with redundancy — running more than one of the thing so the system survives losing one. The standard vocabulary is borrowed from data-center power engineering:

• N — exactly the capacity you need to serve peak load, no spare. If demand is 4 units and each box serves 1, you run 4. Lose any one and you’re now under capacity. N is not redundant.
• N+1 — one spare beyond the need. Run 5 boxes for a 4-unit demand; lose one and the remaining 4 still carry peak. This is the baseline for “redundant,” and it tolerates exactly one failure at a time.
• N+2 — two spares, so you survive a failure while another node is already down (e.g. one out for maintenance and a second dies). Higher availability tiers and anything with slow replacement times want N+2.

The subtlety juniors miss: redundancy must be sized against peak load, not average. “We have two servers, so we’re redundant” is false if a single server can’t carry peak alone — losing one then overloads the survivor, which tips into the queueing collapse from the scalability unit, and your “redundant” pair fails as a unit. True N+1 means each surviving subset can still meet the SLO.

Active-active vs active-passive

There are two ways to arrange redundant copies, and the choice trades cost against recovery speed:

• Active-active — all copies serve traffic simultaneously, behind a load balancer. Lose one and the others absorb its share instantly (you sized for that with N+1). No “switchover” delay, and you’re continuously exercising every replica, so you know they all work. The cost: every copy must be sized and paid for at all times, and stateful active-active needs conflict resolution because two copies can take writes at once.
• Active-passive — one copy serves; the standby sits idle (or warm) and takes over only when the active dies. Cheaper for stateful systems (one writer, no conflicts) and simpler to reason about, but failover takes time, and a standby that’s never exercised may be silently broken when you finally need it — the classic “the backup didn’t work” outage.

The trap: fake redundancy through a shared dependency

Here is the failure mode that turns confident “redundant” architectures into outages: two copies that look independent but share a hidden dependency are not redundant at all. If your two database replicas live on the same physical host, the same storage array, the same rack, the same power circuit, or the same availability zone, then a single event — that host, that array, that rack, that power feed, that AZ — takes both down together. You have paid for two of everything and still have a single point of failure, just a less obvious one.

This is correlated failure, and it is the central reason availability math based on independence (“each replica is 99.9%, so two give 99.9999%”) is optimistic to the point of dangerous. The multiplication only holds if the failures are independent. The moment they share a cause, your real availability is governed by that shared thing, not by the count of replicas. The whole point of cloud availability zones (AZs) is to give you placement domains engineered to fail independently — separate power, cooling, and networking, far enough apart that one flooding/fire/power event can’t take two — so that spreading replicas across AZs makes their failures genuinely uncorrelated.

Blast radius: containing the failure you can’t prevent

You will never eliminate every failure, so the second discipline is bounding how much a single failure can take out — its blast radius. The design question shifts from “will this fail?” (it will) to “when it fails, how much goes with it?” Partitioning helps: shard users across cells so one cell’s failure affects 1/N of users instead of everyone; isolate tenants so a noisy or broken one can’t starve the rest; deploy region by region so a bad release can be caught before it’s global. A system with small blast radii degrades in slices; a system with one giant shared everything fails all at once. Reducing blast radius doesn’t raise the probability of staying up for any single user, but it dramatically lowers the severity of the failures that do happen — and severity is what shows up in the postmortem.