Lock and single-flight: bounding concurrent rebuilds

At a TTL boundary 100,000 requests hit a 50-node fleet. Each node runs an in-process single-flight. How many DB queries happen? Not 100,000 — but not 1 either. The answer reveals exactly where each mitigation layer does and does not help.

Mitigation 1: distributed locking with SETNX

The simplest cross-node mitigation: before running the rebuild, acquire a lock. The Redis primitive is:

SET lock:key uuid EX 30 NX

• NX — set only if the key does not exist (set-if-not-exists).
• EX 30 — auto-expire after 30 s (the safety net for crashed rebuilders).
• uuid — the acquiring process’s unique token (used for fencing, covered in the senior lesson).

What happens at expiry:

1. Request-1 arrives. Runs SET lock:homepage:v1 uuid-A EX 30 NX → success. Starts rebuild.
2. Requests 2–N arrive. Run the same SET → fail (NX). They see the lock is held.
3. Option A: each waiter re-checks the cache on a short sleep (50–200 ms). By the time they re-check, the rebuild may have finished.
4. Option B: each waiter returns a fallback (stale value, default page, empty 204) immediately.
5. Request-1 finishes rebuild. Writes new value. Deletes lock.

The EX=30 is not the cache TTL — it is a safety net. If the rebuilder crashes at step 1 without deleting the lock, the lock auto-expires after 30 s. Set EX to longer than rebuild p99, but short enough that a crash does not stall traffic for too long. A typical target: 3× average rebuild duration.

Mitigation 2: in-process single-flight

Distributed locks coordinate across the fleet; single-flight coordinates within one process. No Redis, no network round-trip — just an in-process map.

The pattern:

type SingleFlight struct { mu sync.Mutex; inflight map[string]*call }

1. Request arrives; cache miss.
2. Check in-process map for key. If a Promise / call already exists → subscribe to it, wait for resolution, return the shared result.
3. If no entry → create a new entry (Promise), start the rebuild, add to map.
4. When rebuild completes → resolve the Promise, remove from map. All subscribers get the result simultaneously.

Go’s standard library ships this as singleflight.Group.Do. Node.js equivalents: p-memoize or a manual Map<key, Promise> pattern.

Cost: O(1) map lookup in process memory. No network. No lock acquire.

Scope: per-process only. A 50-node fleet has 50 independent in-process maps. At a TTL boundary with 100,000 concurrent requests evenly distributed, single-flight alone gives 50 DB queries (1 per node), not 100,000. Add a distributed lock to go from 50 to 1.

Composing both layers

Neither layer is sufficient alone:

• Single-flight only: 50-node fleet still sends 50 concurrent rebuilds.
• Distributed lock only: waiters (all but 1 lock-holder) receive nothing while the rebuild runs — adds latency to every request at the boundary.

Combined stack:

1. Check in-process map → if Promise in flight, subscribe and wait.
2. No in-flight Promise → try SET lock:key uuid EX 30 NX.
3. Lock acquired → register Promise, start rebuild, write value, delete lock, resolve Promise.
4. Lock NOT acquired → retry GET cache after 50 ms. Return stale fallback if still missing.