Read real caching code and config, then reason about behavior: a stale-read race, a miss-rate-to-DB-load calculation, a noeviction misconfiguration, and a hot-key stampede in a read path.
SDSenior◷ 14 min
Level
FoundationsJuniorMiddleSenior
Caching bugs live in the code and config, not the prose: a missing invalidation, a synchronous miss path, a default eviction policy, a hot key read without coalescing. Read each snippet, reason about what it does under load, and pick the answer a senior engineer would commit to.
Practise the loop you run in a caching design or incident review: locate the strategy in the code, predict its behavior under concurrency and load, and choose the change the mechanics actually support.
Snippet 1 — the invalidation order
# cache-aside read + "delete on write" invalidationdef get_user(uid): u = cache.get(f"user:{uid}") if u is None: # miss u = db.read_user(uid) # slow load of CURRENT value cache.set(f"user:{uid}", u) # populate (no TTL) return udef update_user(uid, patch): db.write_user(uid, patch) # new value to DB cache.delete(f"user:{uid}") # invalidate
Quiz
Completed
Under concurrency, a user's stale value occasionally survives an update forever. What is the bug, and the most robust fix?
Heads-up It's correct in isolation but races under concurrency: a slow reader holding the old value can write it back after the delete, and with no TTL it never self-corrects. Versioned keys remove the shared mutable key being raced; a TTL backstop at least bounds the damage.
Heads-up Populating on miss is normal cache-aside. The bug is the interleaving of a slow read's set with the write's delete, plus no TTL to recover. Fix the race (versioned keys) and add a TTL backstop, not the populate step.
Heads-up More nodes don't fix a logical race on a single key — the stale set-after-delete happens regardless of node count. Use versioned keys so the stale write lands on an orphaned old version, and add a TTL so any missed invalidation self-heals.
What do before and after return, and what's the takeaway for the database?
Heads-up You computed reads served from CACHE (hit_ratio × total), not the misses reaching the DB. db_reads = total × (1 − hit_ratio): 800 then 8,000. The DB sees the misses, which jumped 10×.
Heads-up The numbers are right but the conclusion is wrong: 800 → 8,000 is a 10× load increase, not minor. The hit-ratio change looks small precisely because the danger lives on the miss side, which multiplied tenfold.
Heads-up Only hits are absorbed; misses fall through. At 99% there are still 800 misses/sec, and at 90% there are 8,000 — a 10× jump. Compute total × (1 − hit_ratio) to see what the DB actually feels.
Snippet 3 — the Redis cache config
# redis instance used purely as a cachemaxmemory 16gbmaxmemory-policy noeviction # left at default# app issues SET ... EX 600 for every key
Quiz
Completed
What happens when this cache fills its 16GB, and what should the policy be?
Heads-up noeviction protects a durable store by refusing writes under pressure, but a cache is expendable: you WANT it to drop cold keys and keep serving. Failing SETs under load is an outage. Use allkeys-lru/lfu for a cache.
Heads-up noeviction does NOT overwrite anything — it rejects new writes with an error when full. To make room by dropping cold keys you must set an eviction policy like allkeys-lru/lfu.
Heads-up volatile-lru only evicts keys that have a TTL; any TTL-less key is un-evictable and silently fills memory until eviction can't proceed. Since you want any key evictable for a pure cache, use allkeys-lru/lfu.
Snippet 4 — the hot-key read path
# product page for a viral item, recompute is ~300msdef get_page(pid): html = cache.get(f"page:{pid}") if html is None: # miss html = render_page(pid) # expensive: 6 queries + render cache.set(f"page:{pid}", html, ex=30) return html# at peak, ~40,000 concurrent requests for the SAME pid
Quiz
Completed
Every 30 seconds the backend is hit by thousands of simultaneous render_page calls for one pid. What's the failure and the fix?
Heads-up A longer TTL only makes the stampede every 300s instead of 30s — it still detonates on expiry with thousands of simultaneous renders. Coalesce the misses (single-flight) so concurrent readers share one recompute.
Heads-up Faster rendering shrinks each stampede but doesn't stop thousands of simultaneous recomputes of the same value. The structural fix is coalescing concurrent misses into one recompute, regardless of how fast render_page is.
Heads-up A single key can't be split across shards — sharding doesn't help a hot key. The periodic spike on expiry is a stampede; fix it with single-flight (and consider near-caching/replicating the hot key for steady load).
Recall before you leave
01
Why does delete-on-write race, and what's the robust fix?
02
From a hit ratio, how do you compute DB load, and why is the asymmetry dangerous?
03
Why is noeviction wrong for a cache, and how do you stop a hot-key stampede in a read path?
Recap
Every caching decision in this unit shows up directly in the code and config. Delete-on-write invalidation races under concurrency — a slow reader sets the old value back after the delete, and with no TTL it never recovers — so you use versioned keys (and a TTL backstop). The miss-rate math (DB reads = total × (1 − hit_ratio)) shows why 99%→90% hits is a 10× DB load spike: the DB feels the miss side, sized for ~1%. The noeviction default makes a Redis cache fail writes under memory pressure instead of dropping cold keys, so a pure cache wants allkeys-lru/lfu. And a hot-key read path without coalescing stampedes the backend on every expiry — thousands of simultaneous recomputes of one value — fixed by single-flight (one recompute, the rest wait) and optionally probabilistic early recompute. The senior habit is to read the mechanics, predict the behavior under load and concurrency, and choose the structural fix the code supports — not the one (a bigger TTL, more nodes, faster render) that merely papers over it.
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.