Read real load-balancer, gateway, and CDN config, then make the call: a health-check that lies, an algorithm mismatch, a single-instance gateway, and a cache header that leaks or goes stale.
SDSenior◷ 14 min
Level
FoundationsJuniorMiddleSenior
Traffic bugs live in the config, not the prose: a health-check that only opens a socket, an algorithm that balances the wrong thing, a gateway with no replica, a cache header that caches the wrong response. Read each snippet, reason about what really happens under load, and pick the answer a senior engineer would commit to.
Practise the loop you run during a design or config review: locate the load-balancer, gateway, or CDN setting, reason about how it behaves under load and failure, and choose the change the behavior actually supports.
Snippet 1 — the health check
upstream api { server 10.0.1.10:8080; server 10.0.1.11:8080;}# health checklocation /healthz { return 200 "ok"; } # static 200, touches nothing# node 10.0.1.11's DB pool is exhausted; every real request returns 500
Quiz
Completed
Node .11 passes /healthz and stays in rotation while returning 500s to users. Why, and what's the fix?
Heads-up A restart is a one-off; the check will keep passing the next time a dependency fails because it tests nothing. Fix the check to probe the dependency chain so the LB ejects unhealthy nodes automatically.
Heads-up A TCP check is even shallower — it only proves the port accepts a connection. The fix is a deeper HTTP check that touches the DB/cache, not a shallower one.
Heads-up More nodes don't fix a check that can't detect failure; the bad node stays in rotation returning 500s. Make the health check meaningful and add passive ejection on error rate.
Snippet 2 — the algorithm choice
# cache tier: each node holds a hot subset of keys in memoryupstream cache_tier { least_conn; # balance by fewest active connections server cache-1; server cache-2; server cache-3; # ... 12 nodes}# observed: hit ratio is terrible (~10%); almost every request misses
Quiz
Completed
The cache tier uses least_conn and the hit ratio is ~10%. What's wrong, and what algorithm should it use?
Heads-up 10% is catastrophic for a hot cache — it means keys aren't sticking to nodes. The cause is balancing by connections instead of by key; consistent hashing fixes the locality.
Heads-up More nodes make it worse with least_conn — keys scatter even more widely. The problem is the algorithm ignoring the key, not the node count; switch to consistent hashing.
Heads-up Round-robin also ignores the key, so the locality problem stays and the hit ratio stays low. You need key affinity — consistent hashing — not another load-balanced-by-count algorithm.
Snippet 3 — the gateway deployment
# kubernetes deployment for the API gateway (fronts 15 services)kind: Deploymentmetadata: { name: api-gateway }spec: replicas: 1 # single instance strategy: { type: Recreate } # kill old pod, then start new# all 15 services are reachable only through this gateway
Quiz
Completed
What's the risk in this gateway deployment, and what's the minimal fix?
Heads-up Even a fast restart is a full outage of all 15 services while the single pod is down, and Recreate guarantees a gap on every deploy. A single replica is a textbook SPOF; run multiple behind the LB.
Heads-up A StatefulSet doesn't address availability here — the gateway is stateless. The fix is multiple replicas with a rolling update, not stable identities/storage.
Heads-up More resources help throughput, not availability. One bigger pod is still a single point of failure; you need more than one replica with a rolling deploy.
Snippet 4 — the cache header
# response for GET /account/orders (shows the logged-in user's orders)HTTP/1.1 200 OKCache-Control: public, max-age=600# served through a shared CDN; no Vary header
Quiz
Completed
This authenticated, per-user response carries Cache-Control: public on a shared CDN. What's the consequence and the fix?
Heads-up A shared cache keys on the URL, not the user. public + no Vary means it can serve one user's orders to another at the same path. Personalized responses must never be public on a shared cache.
Heads-up A short TTL still serves the cached copy to whoever hits that URL in the window — including a different user. The problem is public on a shared cache, not the freshness window; use private/no-store.
Heads-up Change frequency is irrelevant; the issue is that the response is per-user. Even unchanging data leaks if User B gets User A's cached orders. Mark it private/no-store and Vary on auth.
Recall before you leave
01
Why does a static-200 health check lie, and what makes a check meaningful?
02
When does least-connections/round-robin give a terrible cache hit ratio, and what's the fix?
03
Why is a per-user response with Cache-Control: public on a shared CDN dangerous, and how do you fix it?
Recap
Every traffic decision in this unit reduces to a setting you can read straight off the config. A health check that returns a static 200 (or just opens a socket) lies — it never touches the dependency chain, so a 500-returning node stays in rotation; make it probe the real DB/cache and add passive ejection. An algorithm must match the goal: least-connections/round-robin balance by count and destroy a cache tier’s hit ratio, where consistent hashing on the key preserves locality. A gateway at replicas: 1 is a textbook single point of failure with downtime on every deploy — run multiple replicas behind the balancer with a rolling update. And a per-user response with Cache-Control: public on a shared CDN is a data leak waiting to happen — use private/no-store and Vary. The senior habit is to find the setting, reason about how it behaves under load and failure, and choose the fix the behavior supports — not the one that papers over it.
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.