Concurrency control: combinators, bounded fan-out, and cancellation

The four combinators, and the semantics people get wrong

Before reaching for Promise.all, ask yourself: if one of these tasks fails, do I want the whole batch to fail with it? Your answer determines which combinator is correct — and picking the wrong one is how you either hide partial failures or lose results you already have.

The four static Promise combinators look interchangeable until a failure exposes how differently they settle. Pick the wrong one and you either crash on a partial failure you could have tolerated, or you silently keep results from work you’ve already abandoned.

const settle = (ms, ok = true, v) =>
  new Promise((res, rej) => setTimeout(() => (ok ? res(v) : rej(new Error(v))), ms));

// Promise.all — resolves to an ordered array, OR rejects on the FIRST rejection.
await Promise.all([settle(10, true, "a"), settle(20, true, "b")]);    // ["a","b"]
await Promise.all([settle(10, false, "x"), settle(30, true, "b")]);   // throws Error("x") at 10ms

// Promise.allSettled — NEVER rejects. One descriptor per input, in order.
await Promise.allSettled([settle(10, true, "a"), settle(20, false, "x")]);
// [{ status: "fulfilled", value: "a" }, { status: "rejected", reason: Error("x") }]

// Promise.race — settles as the FIRST to settle, win OR lose.
await Promise.race([settle(50, true, "slow"), settle(10, false, "fast-fail")]); // throws at 10ms

// Promise.any — first FULFILLMENT; if ALL reject, throws AggregateError(.errors[])
await Promise.any([settle(10, false, "x"), settle(20, true, "b")]);   // "b"
await Promise.any([settle(10, false, "x"), settle(20, false, "y")]);  // AggregateError

The mechanism that bites you lives in one sentence: Promise.all rejecting does not cancel the other promises. A promise is not a task you can kill; it is a one-way notification that some already-running work has finished. When Promise.all sees the first rejection it rejects its own promise immediately, but the other fetches, queries, and timers keep running to completion in the background — you’ve simply stopped listening. So a Promise.all over 50 DB calls where call #3 rejects at 10ms still holds all 50 connections open until the slowest of the remaining 47 returns. Use all when partial failure is unacceptable (you want the whole batch or nothing) and allSettled when partial failure is expected and you need to inspect every outcome.

The unbounded fan-out trap

Here is the line that pages people, and the one that looks like the fix but isn’t:

// ANTI-PATTERN A — unbounded fan-out: 48,000 in flight at once
await Promise.all(ids.map(id => fetchFromDb(id)));
// .map runs all 48,000 thunks synchronously → 48,000 pending promises → 48,000
// connection requests slam a pool of 20. Pool drains, queue overflows, ETIMEDOUT,
// and you've DoS'd every service sharing that database. Memory spikes too:
// 48,000 in-flight request objects + their buffers.

// ANTI-PATTERN B — fully sequential: correct, but glacial
for (const id of ids) await fetchFromDb(id);
// One at a time. 48,000 × 40ms = 1,920s ≈ 32 minutes. You traded a crash for a job
// that misses its window. The pool now sits 95% idle while you wait.

Both are wrong, and they are wrong in opposite directions: A asks for infinite parallelism, B asks for none. The senior answer is bounded concurrency — keep exactly N operations in flight at any instant, where N is sized to your downstream capacity (a 20-connection pool wants N ≈ 10–16, leaving headroom for other callers). Throughput is then roughly N / latency: with 40ms calls, N=1 finishes in ~32 min, N=16 in 48000 × 40ms / 16 ≈ 120s, and N=∞ “finishes” by crashing the database. The whole skill is choosing N deliberately instead of letting .map choose ∞ for you.

A bounded worker pool from scratch

You don’t need a library for this, and writing it once cements the mechanism. The pattern: start N “workers” that each pull the next task from a shared cursor and loop until the queue is empty. Tasks are thunks — () => Promise — not already-started promises, because the whole point is to control when each one starts.

async function runPool(tasks, limit) {
  const results = new Array(tasks.length);
  let next = 0; // shared cursor; ++ is atomic between awaits (single-threaded)

  async function worker() {
    while (next < tasks.length) {
      const i = next++;            // claim an index
      results[i] = await tasks[i]();// run one, then loop for the next
    }
  }

  // Start exactly `limit` workers; never more than `limit` tasks are in flight.
  const workers = Array.from({ length: Math.min(limit, tasks.length) }, worker);
  await Promise.all(workers);      // all workers drain the queue
  return results;                  // results stay aligned with input order
}

// Usage: at most 16 DB calls in flight at any moment, regardless of ids.length
const thunks = ids.map(id => () => fetchFromDb(id));
const rows = await runPool(thunks, 16);

Two senior details. First, this is allSettled-unsafe as written — if any tasks[i]() rejects, the await inside worker throws, that worker dies, and the outer Promise.all(workers) rejects (while the other workers keep draining in the background — same non-cancellation rule as above). If partial failure is acceptable, wrap the call: results[i] = await tasks[i]().then(v => ({ ok: true, v }), e => ({ ok: false, e })). Second, in production reach for the battle-tested version — p-limit (wrap each call: limit(() => fetchFromDb(id))) or a Promise.map-with-concurrency from bluebird/p-map. They add the same cap plus niceties (per-call AbortSignal, error aggregation), but the engine is exactly the loop above.

Cancellation that actually cancels: AbortController

A bounded pool stops you from starting too much. AbortController lets you stop work already in flight — the piece Promise.race cannot do. A Promise.race([work, timeout]) settles when the timeout wins, but work keeps running, keeps holding its connection, and may later reject with nobody listening (an unhandled rejection). That’s a leak of the loser. AbortController instead signals the underlying operation to abort for real.

// LEAKY timeout: race resolves at 2s, but the fetch runs to its own 30s+ default,
// still holding a socket, and if it later rejects → unhandledRejection.
const timeout = new Promise((_, rej) => setTimeout(() => rej(new Error("timeout")), 2000));
await Promise.race([fetch(url), timeout]); // fetch is NOT cancelled

// REAL cancellation: the signal tears down the socket at 2s.
const res = await fetch(url, { signal: AbortSignal.timeout(2000) }); // throws TimeoutError; socket closed

// Manual control + combining signals. AbortSignal.any aborts when ANY input does.
const ac = new AbortController();
const signal = AbortSignal.any([ac.signal, AbortSignal.timeout(5000)]);
signal.addEventListener("abort", () => cleanup(), { once: true });
const data = await fetch(url, { signal }); // aborts on ac.abort() OR after 5s
// ...later, e.g. on client disconnect: ac.abort(new Error("client gone"));

The numbers matter for survival under load. Node’s fetch/undici has no overall timeout by default — a stalled upstream can hold a request for tens of seconds (and connection-level defaults are in the 5–10s range, not a request cap), and with a leaky race those sockets pile up until you exhaust the pool, which is the connection stampede again from the outbound side. An AbortSignal.timeout(2000) caps the cost of a single slow upstream at 2s and frees the socket immediately, so a 2s budget instead of a 30s+ hang is a 15× reduction in worst-case occupancy per call. Thread the same signal into every awaited op in a request — fetch, fs reads, your pool’s tasks — so one abort unwinds the whole tree.