open atlas
↑ Back to track
System Design Foundations SD · 03 · 08

Traffic & edge: a load balancer and edge plan

Hands-on project: stand up a real load balancer in front of a small pool, prove that a load-aware algorithm beats round-robin under an uneven node, then design and measure a CDN cache policy — hit ratio, TTL, and what must never hit the shared edge.

SD Senior ◷ 240 min
Level
FoundationsJuniorMiddleSenior

Reading that a load-aware algorithm protects your tail and that a CDN’s hit ratio is non-linear is not the same as proving it on a running system. Stand up a real load balancer in front of a small pool, inject one slow node, and watch round-robin versus least-response-time diverge — then design a cache policy for a mixed app and measure the hit ratio you actually get.

This project makes the whole unit operational: you’ll configure a real balancer, demonstrate the tail-latency difference between a blind and a load-aware algorithm, verify that a meaningful health check ejects a broken node, and then design a per-response cache policy and measure its hit ratio — including proving what must never reach the shared edge.

Project
0 of 8
Objective

Build a small but real traffic edge: a load balancer (nginx/HAProxy/Envoy or a cloud LB) fronting a pool of identical app instances, demonstrate empirically that a load-aware algorithm protects tail latency where round-robin does not, verify health-check ejection, and then design and measure a CDN-style cache policy for a mixed app — closing the loop between the unit's claims and observed behavior.

Requirements
Acceptance criteria
  • A running balancer + pool reachable through one address, with the algorithm and health-check configuration shown.
  • Two load-test results (round-robin vs load-aware) under the same injected slow node, with p50/p99 and throughput, visibly showing round-robin's p99 tracking the slow node and the load-aware algorithm holding the tail near the healthy nodes.
  • A demonstration that breaking a node's probed dependency causes the balancer to eject it (logs or metrics), with traffic continuing on the remaining nodes and recovering when the node is healed.
  • A one-page cache-policy table covering all five content types with explicit Cache-Control/TTL/Vary, and a one-line justification per row, including which responses are never cacheable on a shared edge and why (leak / correctness / write).
  • A measured before/after cache hit ratio for the public content with the change you made, plus the implied origin-RPS reduction (showing you understand the non-linear miss-fraction economics).
  • A short failover write-up: the N−1 headroom calculation and the recommended pool size, tying back to the thundering-herd risk.
Senior stretch
  • Add consistent hashing: put a keyed cache tier behind the balancer, show that hashing on the key gives a high hit ratio while least-connections collapses it, and measure how many keys remap when you add/remove a node (~1/N).
  • Add a thundering-herd scenario: expire a hot object's TTL across many edge/proxy caches at once, observe the synchronized origin spike, then enable request coalescing (and/or an origin shield) and show the origin sees one fetch instead of many.
  • Make the gateway redundant: run two balancer/gateway instances behind a higher-level balancer, kill one mid-load, and show no full outage — contrasting it with the single-instance SPOF.
  • Add jittered retries to the load generator and show how synchronized (un-jittered) retries during a node failure accelerate the cascade, while jitter spreads the recovery load and avoids the brownout-to-outage feedback loop.
Recall before you leave
  1. 01
    How do you empirically show a load-aware algorithm protects tail latency?
  2. 02
    What does a sound CDN cache-policy design specify per response type?
  3. 03
    Why measure the hit ratio and compute N−1 headroom rather than trust the defaults?
Recap

This project turns the traffic-and-edge unit into a repeatable workflow: stand up a load balancer over an identical pool, then prove the unit’s central claim by injecting one slow node and showing round-robin pins the service p99 to it while a load-aware algorithm (least-connections/least-response-time) routes around it. Verify that a meaningful health check — one that probes a real dependency, not a static 200 — ejects a broken node and that traffic continues. Then design a per-response CDN cache policy for a mixed app (fingerprinted assets, public page, per-user page, must-be-correct value, write endpoint), measure the hit ratio before and after a TTL/cache-key improvement, and translate it into the non-linear origin-load reduction. Finally, do the N−1 failover capacity math so the survivors can absorb a lost node without a thundering-herd cascade. Doing the configuration and measurement, not just reading the claims, is the whole discipline — an engineer who has built this once chooses layers, algorithms, gateways, and cache policies deliberately instead of discovering their failure modes during an incident.

Something unclear?

Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.

shortcuts expand
search
K
prev piece
k
next piece
j
cycle tier
t
this menu
?
sources3
expand
  1. 01
  2. 02
  3. 03

Trademarks belong to their respective owners. Editorial reference only.