Dogpile effect: the concurrency collision at the instant a hot key expires

3am page. The homepage feed query — a 200ms aggregation that normally runs maybe twice a minute — is suddenly running thousands of times a second. The database is at 100% CPU and the connection pool is exhausted. Nothing was deployed. Then someone reads the cache config: the feed key has a 5-minute TTL, and at 02:55:00 it expired. At that instant ~5000 requests were in flight, every one of them missed the same key, and all 5000 went to the database at once. One expired key, 5000 simultaneous identical queries.

Dogpile is the concurrency-at-expiry angle

There is a broader topic — cache stampede / thundering herd — about a missing key flooding the origin. The dogpile effect is the sharpest, most specific face of it: the collision that happens precisely at the moment of expiry on a single hot key. The hotter the key, the worse the dogpile, because hotness means many requests are concurrently in flight, and a TTL expiry is an event that hits all of them in the same millisecond.

Walk the timeline. The key is cached and serving thousands of reads per second straight from memory — the origin is idle. The TTL clock ticks to zero. Request #1 reads the cache, gets a miss, and starts recomputing. While it is still recomputing (it takes 200ms), requests #2 through #5000 arrive, each reads the cache, each also gets a miss — because #1 has not written the new value yet — and each also starts recomputing. The gap between “first miss” and “value rewritten” is the entire window in which the dogpile forms. A naive cache-aside read has no coordination across those concurrent misses, so the origin goes from idle to N-parallel in one tick.

The cost math is the whole problem

The reason the dogpile is dangerous is multiplicative, and the numbers are unforgiving. Take an expensive operation — say a 200ms database aggregation. At steady state behind a warm cache, it runs almost never. At the expiry instant, it runs once per concurrent request. One expensive query × N concurrent requests = origin meltdown.

Plug in real traffic. The Wikipedia framing: a page that takes 3 seconds to render at 10 requests per second means 30 processes recomputing simultaneously the moment it expires. Now scale it: 5000 requests/sec hitting a 200ms query means up to 5000 simultaneous database connections, each holding the query for 200ms — far past most pool limits (a Postgres pool is often 20–100 connections). The pool exhausts, healthy unrelated queries queue behind the herd, and a cache miss on one key takes the whole database down. The query cost was never the issue at steady state; the concurrency multiplier at the expiry instant is.

Single-flight: collapse N misses into one recompute

The first reflex is a lock. On a miss, the first request acquires a lock for that key and recomputes; everyone else who misses during the window blocks on the lock and, when it’s released, reads the freshly written value instead of recomputing. Go’s golang.org/x/sync/singleflight is the canonical implementation: call Do(key, fn) and if a call for that key is already in flight, additional callers wait and receive the original call’s result rather than re-executing fn. Reported impact is dramatic — Reddit cut database peak load ~3× with request coalescing on feed endpoints, and flooding a server with 100 concurrent requests for one key can deduplicate ~95% of the redundant expensive runs, turning many 3–5s waits into one 3–5s run shared by all callers.

The senior caveat: a local mutex only coalesces within one process. With 20 app instances behind a load balancer, each instance runs its own single-flight, so the origin still sees up to 20 concurrent recomputes — better than 5000, but not one. To get one recompute across the fleet you need a distributed lock (a Redis SET key val NX PX 30000, for example), which buys cross-process coordination at the cost of a network round-trip and a new dependency on the lock store’s availability.

XFetch: never let the hot key expire under load

Locking fixes the symptom after the miss; probabilistic early expiration (XFetch, from the VLDB 2015 paper “Optimal Probabilistic Cache Stampede Prevention” by Vattani, Chierichetti, and Lowenstein) attacks the cause — it makes the key practically never expire while it is hot. Each reader, on a hit, rolls a die: it voluntarily recomputes early with a probability that rises as expiry approaches. The decision rule is:

time() − delta * beta * log(rand(0,1)) ≥ expiry

where delta is how long the last recompute took, beta is a tuning knob (the paper proves beta = 1 is effectively optimal), and log(rand(0,1)) draws from an exponential distribution. The elegance: a more expensive value (large delta) gets refreshed earlier and more eagerly, and because each reader decides independently, the first lucky reader recomputes alone — well before the TTL hits zero — and rewrites the key, so when the herd arrives the value is fresh and nobody else misses. The collision is dissolved because the expiry event no longer coincides with peak concurrency.

Senior failure modes: the cure that becomes the disease

Each fix introduces its own production hazard, and the dogpile postmortems are usually about the fix failing, not the original miss.

The recompute lock can become a serialization bottleneck. If the recompute is slow and the key is extremely hot, every reader now blocks on one lock holder for the full recompute duration — you have converted a parallelism storm into a latency cliff where p99 for that endpoint becomes the recompute time for everyone. Worse is a lock without a timeout: if the single recomputer crashes or hangs mid-flight while holding the lock, every reader blocks forever — a self-inflicted total outage where the database is healthy and idle but every request is deadlocked waiting on a lock that will never be released. Always set a lock TTL (and make it longer than the worst-case recompute, or the lock expires mid-recompute and you re-open the dogpile). TTL jitter is real but narrow: it stops a batch of keys cached together from expiring in the same second, but it does nothing for one individual hot key — that key still has exactly one expiry instant and its own herd waiting on it. Reaching for jitter to fix a single-hot-key dogpile is a common misdiagnosis.