Multiple-choice synthesis across the scalability unit: Little's Law and the queueing curve, vertical vs horizontal tradeoffs and the USL, back-of-envelope arithmetic, and the latency numbers used to reject designs.
SDSenior◷ 13 min
Level
FoundationsJuniorMiddleSenior
Six questions that cut across the whole unit. Each is a decision you make when you size a service, review a design doc, or estimate a system on a whiteboard — not a definition to recite, but the arithmetic of Little’s Law, the USL, and the latency numbers under real load.
Confirm you can apply Little’s Law to find a throughput ceiling, read the queueing curve, place a workload on the vertical/horizontal axis with the USL in mind, run a back-of-envelope estimate to one significant figure, and reject a design from the latency numbers alone.
Quiz
Completed
A service is capped at 150 concurrent in-flight requests by its connection pool, and each request averages 30 ms end to end. By Little's Law, what is its maximum sustainable throughput?
Heads-up That ignores W. Each slot turns over every 30 ms, serving ~33 requests/sec, not 1. λ = L / W = 150 / 0.030 = 5,000 RPS.
Heads-up You multiplied concurrency by latency in ms. Little's Law divides, with W in seconds: 150 / 0.030 = 5,000 RPS.
Heads-up Little's Law needs only L and W — it's an accounting identity with no hardware assumption. λ = L / W = 5,000 RPS.
Quiz
Completed
At 90% utilization your p99 latency is ~10× the unloaded service time, and the team wants to push to 98% to 'use the hardware fully.' What does the queueing curve predict?
Heads-up Queueing delay is non-linear: W = 1/(μ−λ). Going from 90% to 98% roughly quintuples the tail (10× → ~50× unloaded), not a linear 9% bump. That's the knee of the curve.
Heads-up The tail grows steeply well before 100%. At 90% it's already ~10× unloaded; at 98% ~50×. The whole point of the curve is that latency explodes near the limit, not at it.
Heads-up You may gain a little throughput, but at a latency cost that's non-linear and often violates SLOs. Mature systems trade that last slice for headroom precisely because the tail goes vertical.
Quiz
Completed
A stateless API tier scales out linearly to ~30 nodes, then total throughput plateaus and finally dips slightly past ~45 nodes. By the Universal Scalability Law, what's happening?
Heads-up Contention (α) makes throughput plateau at a ceiling, but it never goes DOWN. A throughput that falls as you add nodes is the signature of the coherency term β, which grows quadratically.
Heads-up It can, and that's the USL's β term: past a peak, coordination cost exceeds the work each node adds, so the cluster slows. A 50-node cluster can trail a 30-node one.
Heads-up More nodes don't inherently raise per-request latency; the symptom is falling AGGREGATE throughput, which the USL attributes to coherency β, not to latency.
Quiz
Completed
Estimating to one significant figure: 50M daily active users each make 20 requests/day. What is the average QPS, and what peak should you provision for?
Heads-up That's requests per DAY (10^9), not per second. Divide by ~86,400 s/day (≈10^5): 10^9 / 10^5 = ~10,000 QPS average. Then apply a peak multiplier.
Heads-up The average is right, but sizing for the average is the classic error — traffic is bursty. Provision for the peak (~3× average) so you survive the evening spike.
Heads-up You divided twice. Requests/day = 50M × 20 = 10^9; ÷ 86,400 ≈ 10,000 QPS. Don't divide by the per-user rate again — it's already in the 10^9.
Quiz
Completed
A design doc proposes serving 50,000 reads/second by random-seeking records from a spinning HDD. Using the latency numbers, what is the verdict, with the binding limit?
Heads-up The design says RANDOM seeks, which are bound by the ~10 ms seek time (~100/sec), not sequential bandwidth. The sequential-vs-random gap is the trap; the design needs ~500 disks.
Heads-up Record size affects bandwidth, but the binding limit is the seek RATE: ~100 random seeks/sec/HDD regardless of size. 50K/sec fails by ~500×.
Heads-up Correct verdict, wrong ratio: RAM random reads are ~10^5× faster than an HDD seek, and the disk's limit is ~100 random seeks/sec. The factor is enormous, not 10×.
Quiz
Completed
Your single write-primary database is at 70% CPU on the second-largest instance type, and traffic grows 8%/month. Which plan reflects the vertical-scaling lesson?
Heads-up There isn't — you're one step from the largest instance, with nowhere to go after that. Discovering the ceiling during an outage is the exact failure the vertical-scaling lesson warns against.
Heads-up Sharding a write primary is a major migration you start deliberately, not under fire. The right move is a stopgap bump WHILE you begin scale-out, before you hit the ceiling.
Heads-up At 8%/month the remaining headroom is ~4–6 months, and you're near the top instance. Waiting guarantees you hit the ceiling with no runway for the migration.
Recall before you leave
01
Why keep utilization at 60–70% instead of pushing to 98%?
02
How do you reject a 50K-reads/sec spinning-disk design in one sentence?
Recap
The through-line is that scaling is arithmetic you do before you build. Little’s Law (λ = L/W) gives a throughput ceiling from concurrency and latency; the queueing curve (W = 1/(μ−λ)) makes the tail go vertical near the limit, so you keep 30–40% headroom instead of squeezing to 98%. Vertical scaling is simple but capped, superlinear in cost, and a SPOF — so you bump as a stopgap while starting scale-out before the ceiling; horizontal scaling is elastic but pays the USL coordination tax (α plateaus, β makes the curve retrograde). Back-of-envelope estimates (requests/day ÷ ~10^5, sized for a 3× peak, rounded to one significant figure) tell you which resource breaks first, and the latency numbers (~100 random seeks/sec on HDD, ~150 ms cross-region) let you reject a bad design in a sentence — no benchmark required.
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.