Cache invalidation: the set-after-delete race and the consistency you actually buy

A user changes their display name. The PUT /users/{id} returns 200, the row in Postgres is updated, and the handler dutifully runs DEL user:42 to drop the cache. They refresh — and the old name is back. Not for a second: for the next twelve hours, until the TTL expired. The on-call engineer could not reproduce it locally because it needs two requests racing by a few milliseconds. The write deleted the key; a read that started before the write, holding the old DB row, finished after the delete and wrote the stale value back. The cache was lying, and the database was telling the truth, and they disagreed for half a day.

Invalidation is the choice of which staleness to ship

There is no cache without staleness; there is only the staleness you chose and the staleness that surprised you. Every invalidation strategy is a different answer to one question: when the source of truth changes, how does the copy find out? TTL waits for a clock. Event-driven purge listens for the write. Write-through updates both in the same call. Write-behind updates the cache now and the database later. Each buys a different consistency guarantee and pays in a different failure mode — there is no strategy that is fresh, fast, and simple at once.

The reason this is hard is that a cache and a database are two systems, and you are trying to keep two systems agreeing without a transaction spanning both. The moment a write touches one and not yet the other, a reader can observe the gap. Naming the strategies is easy; the senior work is knowing which gap each one opens and whether your product can tolerate it.

TTL: cheap, eventually correct, and synchronized by accident

Time-to-live is the floor of caching. You stamp each entry with an expiry; reads serve it until the clock runs out, then the next read repopulates. It needs no write path coordination at all — the database does not even know a cache exists. The cost is bounded staleness: with a 60-second TTL, a changed value can be wrong for up to 60 seconds, and there is nothing you can do about it short of shortening the TTL (more origin load) or adding active invalidation on top.

The production trap with TTL is not the staleness — it is synchronized expiry. If 1,000 app servers all cache the same hot key with an identical 300-second TTL, and they all warmed it within the same deploy window, they all expire it within the same ~100ms window. At T=300 every one of them misses simultaneously and slams the origin in lockstep — the thundering herd / cache stampede. The fix is TTL jitter: instead of a fixed 300s, set each entry to a random value in, say, 270–330s (±10%). The expiries scatter across a 60-second band, the origin sees a smooth ramp instead of a spike, and a single rebuild serves the rest. Facebook’s memcache work and Redis both treat jitter as the baseline defense; the more aggressive version is probabilistic early refresh, where each read past a threshold has a small chance of refreshing early, turning a sharp 100ms spike into a 60-second ramp.

Event-driven purge and the dual-write problem

If bounded staleness is not good enough, you invalidate on the write itself: when the row changes, delete (or update) the cache key. This is the “cache-aside with purge” pattern, and it is where the dragons live. You now have two writes — one to the database, one to the cache — and no transaction across both. That is the dual-write problem: either store can succeed while the other fails. If the DB commit succeeds and the DEL fails (a dropped packet, a Redis blip), the cache serves the old value until its TTL — which is why you keep a TTL even when you actively invalidate: it is the backstop for every missed purge.

Deleting rather than updating the key is itself the senior reflex: a DEL is idempotent and order-independent, so two concurrent writers racing to invalidate cannot leave a half-written merged value the way two concurrent SETs can. But delete-on-write does not close the worst race, which is the next section.

The set-after-delete race: why “just delete the key” still serves stale

Here is the failure from the hook, in order. Reader R gets a cache miss on user:42 and queries the database, reading name = "Ann". Before R writes that back, writer W runs: it updates the row to name = "Bob" and deletes user:42. Now R — still holding the stale “Ann” it read milliseconds ago — finishes and does SET user:42 = "Ann". The cache now holds “Ann” while the database holds “Bob”, and it stays wrong until the TTL expires. The delete happened between R’s read and R’s write, so R’s write silently undid it.

There are three real fixes, in rising order of cost. Delayed double delete: after the write, DEL the key, then schedule a second DEL a few hundred milliseconds later — long enough that any in-flight reader has finished its stale SET, so the second delete wipes it. Cheap, probabilistic, and the most common patch in practice. Leases (Facebook’s approach): on a miss, memcache hands the reader a 64-bit token bound to the key; the reader may only SET if its token is still valid, and a DEL invalidates the token — so a stale set arriving after the delete is rejected. Leases cut Facebook’s peak origin load from 17,000 to 1,300 queries/sec on a hot key while also killing this race. Funnel writes through the cache (write-through), so there is no separate reader-populated SET to race with at all.

Write-through, write-behind, write-around: where the write goes

The strategies above are read-populated (cache-aside). The write-path strategies decide what a write does to the cache directly. Write-through writes to the cache and the database in the same operation, synchronously — the cache is always consistent with what was just written, and there is no reader-populated stale set to race. The price is write latency (you pay both stores on the critical path) and cache pollution: you cache data that may never be read. Write-behind / write-back writes to the cache immediately and queues the database write asynchronously — fast writes, write coalescing under bursts, but a window where the only copy of the write is in a volatile cache. If the node dies before the flush, that write is gone; this is the strategy that can lose committed-looking data, so it is reserved for tolerant workloads (counters, metrics) or paired with a durable queue. Write-around skips the cache on write entirely (DB only) and lets reads populate later — good when written data is rarely read soon, bad for read-your-writes because the writer’s own next read is a guaranteed miss-then-stale-until-populated.