Caching
SWR: flatten the revalidation sawtooth
Reading about the p99 sawtooth is not the same as making it disappear from a real latency graph. Stand up a cache in front of a slow origin, drive it into the per-TTL spike, then apply stale-while-revalidate, stale-if-error, and single-flight until the tail goes flat — with evidence at every step.
Turn the unit’s mental model into a reproducible loop: measure the synchronous-revalidation spike, decouple freshness from latency with the SWR window, defend against an origin outage with stale-if-error, coalesce the background refresh, and verify the flat tail with before/after numbers.
Put a cache (a CDN edge, a reverse proxy like Varnish/nginx/Fastly, or a small in-process cache you write) in front of a deliberately slow origin, reproduce the per-TTL p99 sawtooth, then flatten it with stale-while-revalidate and stale-if-error plus single-flight — proving each change with measurements under identical load.
- A before/after comparison under identical load: p99 latency over time for max-age-only versus max-age+SWR, with the sawtooth visibly gone in the after graph — measured, not estimated.
- An origin-request-rate panel showing background refreshes coalesced to roughly one per key per expiry after single-flight, versus the stampede before it.
- A demonstration that during a forced origin 5xx the cache serves stale (stale-if-error) and that past the configured window it stops serving stale and surfaces the failure — proving staleness is bounded.
- A one-paragraph write-up: the maximum staleness your final policy allows, why that bound is acceptable for this data, and the one endpoint you deliberately excluded from SWR and why.
- Reproduce the same pattern on the client with the SWR (or React Query) library: render a component stale-first, show dedupingInterval coalescing two concurrent calls into one request, and revalidateOnFocus refreshing a stale tab.
- Add an on-call runbook: how to read the sawtooth on a dashboard, the three header windows and what each defends, the single-flight and jitter knobs, and the bounded-staleness checklist.
- Add a freshness-policy lint to CI that fails the build if any auth, permission, or feature-flag endpoint ships a stale-while-revalidate or stale-if-error directive.
- Measure the staleness distribution: instrument how old served responses actually are under steady load and confirm it stays inside your configured window — turning 'bounded staleness' from a claim into a measured fact.
This is the loop you will run on every real freshness incident: measure the synchronous-revalidation sawtooth first, decouple freshness from latency with stale-while-revalidate so no human waits on the refresh, coalesce the background refresh with single-flight plus jitter so a hot key cannot stampede the origin, defend an outage with a bounded stale-if-error floor, and carve auth out of the policy entirely. Doing it once against a toy origin makes the production version — flattening a real p99 graph without breaking correctness — muscle memory.