Bottlenecks & tradeoffs
Find the bottleneck — the one resource that breaks first — and state every tradeoff: consistency vs availability, latency vs cost, simplicity vs scale. Handle 'what if 10x?' by naming which resource breaks and why. Acknowledging the tradeoff separates a mid answer from senior.
“What if traffic grows 10×?” The mid-level candidate answers “we’d add more servers” — true, and almost meaningless, because it doesn’t say which part falls over first. The senior candidate pauses, walks the request path, and says: “the stateless tier scales out fine, so that’s not it. At 10× the write rate hits ~100K/sec, which is past even a sharded primary’s comfortable range, so the bottleneck moves to the write path — and to keep latency we’d accept eventual consistency on the read replicas, trading freshness for availability. The cost is reads can be a few seconds stale.” Same question. One answer names a resource and the price; the other names neither. That gap is the seniority signal.
The bottleneck is a single resource
A system’s capacity is set by its bottleneck — the one resource that saturates first as load rises. Like a chain breaking at its weakest link, the whole system stalls when any single resource hits its ceiling, no matter how much headroom the others have. Adding capacity anywhere else is wasted; you must find and widen the binding constraint.
So when the design is drawn, walk the request path and ask, at each hop, what is this component’s ceiling, and how close is the load to it? The candidate resources are always the same short list:
- CPU on the service tier — usually the easiest to fix (scale out the stateless tier).
- The database — the write primary’s throughput, or a read replica’s, or a single hot shard. Most often the real bottleneck, because state is hard to scale (the vertical-vs-horizontal lesson).
- A hot key or hot shard — uneven load where one partition gets disproportionate traffic, so the average utilization lies.
- Network/bandwidth — egress on a fan-out or media-heavy path.
- A downstream dependency — a third-party API, a shared lock, a single coordinator (the USL contention term).
Together, these candidates form the full list of places a system can buckle — but without the estimate in hand, the list is just trivia. The skill is not listing these; it’s identifying which one is binding for this design at this load, using the estimate from earlier. The bottleneck is the resource your back-of-envelope number is closest to exceeding.
▸Why this works
Why does the bottleneck dominate, and why is the average utilization a liar about it? Because the bottleneck is set by the worst-loaded resource, not the mean. A database cluster at 40% average CPU can be melting if one shard holds a celebrity’s data and runs at 99% — the average hides the hot shard, exactly as the average latency hid the tail in the latency-vs-throughput lesson. So you don’t look for “the busy tier”; you look for the single hottest resource, which is often a hot key, a hot partition, or one synchronous dependency in an otherwise parallel path. Find that, and you’ve found what actually caps the system — and widening anything else first is motion without progress.
Every design is a set of tradeoffs
There is no free architecture. Every choice buys one property by spending another, and the senior signal is naming the trade explicitly rather than pretending a design is strictly better. The recurring axes:
- Consistency vs availability (the CAP/PACELC trade). Strong consistency means coordination, which costs availability under partition and latency always (PACELC’s “else, latency”). Eventual consistency buys availability and low latency at the price of stale reads. State which you chose and why this domain tolerates it: “a like count can be eventually consistent; a bank balance cannot.”
- Latency vs cost. Lower latency usually means more replicas, more cache, more regions — all money. You can hit almost any latency target by spending enough; the design question is the cheapest way to hit the target you actually need.
- Simplicity vs scale. The simplest design that meets the requirements is usually right; complexity should be added only where an estimate forces it. A single Postgres instance that fits the load beats a sharded, multi-region system you didn’t need (the over-engineering trap, again).
- Read vs write optimization. You can precompute on write to make reads cheap (fan-out-on-write) or compute on read to make writes cheap (fan-out-on-read) — but not both. The access pattern’s read/write ratio decides which.
These four axes cover virtually every architectural decision you’ll make. Without naming the tradeoff on each one, your design is a set of assertions; with it, it’s engineering. The move is to say the trade out loud: “I’m choosing eventual consistency here, which buys availability and low read latency; the cost is reads can be a few seconds stale, which is acceptable for a timeline but I’d revisit it for payments.” That single sentence is the difference between an answer and a strong answer.
Handling “what if 10×?”
The scaling question is the interviewer’s favorite probe, and it tests exactly the bottleneck skill. The wrong answer is a reflexive “add more servers.” The right answer is a procedure:
- Walk the path and find the new bottleneck. At 10× load, which resource crosses its ceiling first? Recompute the key estimate (the estimation lesson): 10× the QPS, 10× the storage growth. The stateless tier usually scales out trivially, so the bottleneck almost always moves to a stateful resource — the write primary, a hot shard, a cache that no longer fits in RAM.
- Name the fix for that specific resource, and the new tradeoff it introduces. “Sharding the write tier 10× means cross-shard queries get expensive, so I’d denormalize the common read.” Each fix moves the bottleneck somewhere new — say where it goes next.
- State what breaks if you do nothing, with the mechanism: the queueing curve goes vertical, retries pile on, and a brownout cascades to an outage (the scalability and cascading-failure lessons).
What separates mid from senior
The same design, presented two ways, lands completely differently. The mid-level engineer presents the design as a set of correct choices. The senior engineer presents the design as a set of deliberate tradeoffs, each with its cost named and its alternative acknowledged. The technical content can be identical — the difference is metacognition: knowing what you gave up, when you’d revisit it, and under what load it breaks.
Concretely, the senior signals are: identifying the binding bottleneck (not “it’s slow” but “the write primary at 8K/sec is the constraint”); stating each tradeoff with its price (“eventual consistency buys availability, costs freshness”); knowing the failure mode (“past 70% the queueing tail goes vertical”); and right-sizing (“a single instance is enough here; I’d shard only past ~10K writes/sec”). Certainty is a junior tell; calibrated tradeoff-awareness is the senior one.
▸Common mistake
The most common tradeoff mistake is presenting a design as strictly better with no downside — “we’ll use a cache, which makes it faster,” full stop. The interviewer immediately knows you haven’t thought it through, because a cache also introduces invalidation complexity, a consistency window, a cold-start thundering herd, and a new failure mode (what happens when the cache is down?). Every component you add buys something and costs something. The fix is a reflex: after every design choice, ask yourself “and what did that cost me?” and say the answer out loud. An engineer who volunteers the downside of their own decision is far more credible than one who has to be cornered into admitting it.
The interviewer asks 'what if traffic grows 10×?' Which response shows senior-level bottleneck reasoning?
A database cluster shows 40% average CPU, yet users report timeouts. What's the most likely bottleneck, and why does the average mislead?
What separates a senior answer from a mid one is naming the _______ explicitly: stating that a choice (say, eventual consistency) buys one property (availability, low latency) at the cost of another (stale reads), rather than presenting the design as strictly better with no downside.
- 01How do you find the bottleneck, and why does average utilization mislead?
- 02Name the recurring tradeoff axes and how you state one well.
- 03What's the procedure for a '10× traffic' question, and what's the senior vs mid difference?
A system’s capacity is set by its bottleneck — the single resource that saturates first — so you walk the request path and find which component’s ceiling the load is closest to, using the estimate from earlier. The candidates are always CPU (easy, scale out the stateless tier), the database (usually the real one), a hot key or hot shard (where the average utilization lies — a 40% cluster can be melting on one shard at 99%), network, or a downstream dependency. Then state every choice as a deliberate tradeoff with its price named: consistency vs availability (the CAP/PACELC trade — “a like count can be stale, a bank balance can’t”), latency vs cost, simplicity vs scale, read- vs write-optimization. Handle “what if 10×?” as a procedure — recompute the estimate, find the resource that now breaks first (almost always stateful), name the targeted fix and its new tradeoff, and say where the bottleneck moves next and what cascades if you do nothing. The through-line, and the whole point of this unit, is the seniority signal: certainty is a junior tell; naming the binding bottleneck, the price of each tradeoff, the failure mode, and the right size is the senior one. Volunteer the downside of your own decision before you’re cornered into it. Now when you hear “what if 10×?” or “why did you add that?”, your first move is to walk the request path, find the single resource that breaks first, and name its price — not reach for a pattern name.
Practice
Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.
Apply this
Put this lesson to work on a real build.