Scalability: numbers and config reading
Read real capacity math, config, and code, then do the arithmetic: a Little's Law ceiling, a queueing budget, an estimation helper, and a latency check that rejects a design.
Scaling bugs live in the numbers, not the prose: a pool size, a utilization target, a rounding step, a per-request call to a far-away region. Read each snippet, do the arithmetic in your head, and pick the answer a senior engineer would commit to.
Practise the loop you run during a design or capacity review: locate the numbers in the code, apply Little’s Law or the queueing curve or the latency hierarchy, and choose the change that the arithmetic actually supports.
Snippet 1 — the pool sizing helper
def max_throughput_rps(pool_size: int, avg_latency_ms: float) -> float:
# Little's Law: λ = L / W, with W in seconds
return pool_size / (avg_latency_ms / 1000)
print(max_throughput_rps(pool_size=200, avg_latency_ms=40)) # what prints?What does max_throughput_rps(200, 40) return, and what does it mean?
Snippet 2 — the utilization target
# autoscaler config
target_cpu_utilization: 0.95 # "use the hardware fully"
# unloaded service time ~20 ms; SLO is p99 <= 200 msThe team set the autoscaler target to 95% CPU to save cost. Using the queueing model, what's the risk and the fix?
Snippet 3 — the estimation helper
def daily_to_qps(daily_requests: float, peak_multiplier: float = 3) -> dict:
avg = daily_requests / 86_400
return {"avg_qps": round(avg), "peak_qps": round(avg * peak_multiplier)}
# 1.5 billion requests/day
print(daily_to_qps(1_500_000_000))Roughly what does daily_to_qps(1_500_000_000) return, and what's the right way to read it?
Snippet 4 — the per-request call
# service runs in us-east-1 (Virginia); this store is in eu-central-1 (Frankfurt)
def handle_request(req):
user = remote_store.get(req.user_id) # Virginia -> Frankfurt -> Virginia
prefs = remote_store.get(req.prefs_id) # Virginia -> Frankfurt -> Virginia
return render(user, prefs) # two synchronous cross-region callsUsing the latency numbers, what is this handler's latency floor, and what's the right fix?
- 01Given a pool size and average latency, how do you compute the throughput ceiling, and what's the unit trap?
- 02Why is a 95% CPU autoscaler target dangerous, and what target holds an SLO?
- 03What's the latency floor of two synchronous cross-region calls in series, and the fix?
Every scaling decision in this unit reduces to arithmetic you can read straight off the config. Little’s Law (λ = L/W, with W in seconds) turns a pool size and latency into a throughput ceiling — and the unit trap is forgetting to convert milliseconds to seconds. The queueing curve (W = 1/(μ−λ)) says a 95% utilization target blows a tight p99 SLO because the tail is ~20× unloaded there, so you set ~60–70%. Back-of-envelope code (daily ÷ 86,400, then × a peak multiplier, read to one significant figure) gives the load you size for — the peak, never the average. And the latency numbers (~150 ms per cross-region round trip, ~100 random seeks/sec on HDD) let you reject a per-request cross-region call or a disk-per-request read in a sentence. The senior habit is to find the numbers, do the math, and choose the fix the arithmetic supports — not the one that papers over it.
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.