Backpressure and bounded concurrency

A migration script reads a million user IDs and “for each, fetch the profile and upsert it.” Written the obvious async way — await Promise.all(ids.map(processUser)) — it launches a million concurrent operations in the same instant. The database opens connections until it refuses, memory climbs as a million pending promises and their closures pile up, and the process is OOM-killed before the first thousand finish. The code was correct. The concurrency was unbounded, and an event loop will happily start infinite work it can never finish.

The loop will not stop you

The event loop’s gift — start thousands of operations cheaply — is also a loaded gun. Nothing in Promise.all(items.map(fn)) limits how many fns are in flight; it starts all of them, immediately. For ten items that is fine. For a million it is a self-inflicted denial of service: every in-flight operation holds memory (buffers, closures, a pending promise), and every one hammers whatever it calls. Two distinct disciplines tame this: backpressure (let a slow consumer push back on a fast producer) and bounded concurrency (cap how many operations run at once). They solve the same disease — producing faster than you consume — at different layers.

Backpressure: the consumer pushes back

Backpressure is a feedback signal from a slow consumer to a fast producer: “stop sending, I’m full.” Node streams have it built in. When you writable.write(chunk) and the internal buffer exceeds the highWaterMark (default 16 KB for byte streams, 16 objects in object mode), write() returns false — your cue to stop writing until the stream emits a drain event signaling the buffer has emptied. Honor that signal and memory stays bounded; ignore it and Node keeps buffering every chunk in memory until the process OOMs.

The reason pipe() and pipeline() are the recommended way to connect streams is that they wire this handshake automatically — pausing the readable when the writable is full, resuming on drain — so a 50 GB file copies through a few hundred KB of live buffer instead of loading whole. Manual read/write loops that skip the false/drain dance are the classic source of streaming OOMs.

Bounded concurrency: cap the fan-out

Streams handle the byte-pipeline case. The other case is N independent async tasks — fetch these 10,000 URLs, process these million rows — where the fix is a concurrency limit: run at most k at a time, and as each finishes, start the next. The migration above becomes safe by replacing Promise.all(ids.map(fn)) with a limiter (e.g. p-limit(20)) so only 20 operations are ever in flight, or by using a worker-pool/for await pattern that pulls from the list as capacity frees up.

The numbers make the case: a sequential for...of await loop and a bounded Promise.all can differ by orders of magnitude in wall time (one benchmark: ~30 s sequential vs ~340 ms concurrent), but unbounded concurrency does not beat bounded — past the point your downstream can absorb, extra in-flight work only adds queueing, memory, and failures. The sweet spot is the largest k the downstream tolerates, not infinity. That k is usually set by the downstream’s own limit: a connection pool of 20, an API rate limit, a disk’s IOPS.

Three regimes, one decision

Picture the spectrum. Sequential (await in a loop) is one-at-a-time: safe, slow, no pressure on anything. Unbounded (Promise.all over a huge list) is all-at-once: fast to start, catastrophic at scale. Bounded (limit k) is the production answer: fast enough, predictable memory, downstream-respecting. The senior reflex on seeing Promise.all(bigArray.map(...)) is immediate: what bounds this? If the array size is attacker- or data-controlled, unbounded Promise.all is a latent outage.