Worker threads and clustering

The single-thread wall

Node runs your JavaScript on one thread. That is a feature for I/O: an await fetch() or await db.query() parks the work in libuv or the kernel and the event loop moves on, so one thread juggles thousands of concurrent connections at near-zero cost. The model breaks the instant work is CPU-bound — hashing a password with bcrypt, transforming an image, gzipping a payload, parsing a 20 MB JSON body, or a JSON.parse on a huge string. None of that yields to the loop. While it runs, the loop is blocked: no callbacks, no timers, no new requests, for every client, not just the one whose request triggered it.

The official guidance — Don’t Block the Event Loop — is the whole game. A 200 ms synchronous CPU task isn’t “200 ms slower for that user”; it adds 200 ms of head-of-line latency to every request already queued behind it. There are exactly two escapes, and they map to two different problems:

• More threads in one process — worker_threads — for CPU parallelism with optional shared memory.
• More processes — cluster, or N replicas behind a load balancer — to use all cores for a network server.

Neither is for I/O. If your endpoint is slow because it awaits a database, a worker won’t help — the bottleneck is the database, and the loop was never blocked.

worker_threads: real threads, isolated memory

worker_threads gives you real OS threads inside one Node process, each with its own V8 isolate and event loop. You spawn one with a script and hand it data; it computes and posts a result back.

// main.js
import { Worker } from "node:worker_threads";

function hashInWorker(password) {
  return new Promise((resolve, reject) => {
    const worker = new Worker("./hash-worker.js", { workerData: password });
    worker.on("message", resolve);
    worker.on("error", reject);
    worker.on("exit", (code) => {
      if (code !== 0) reject(new Error(`worker exited ${code}`));
    });
  });
}

// hash-worker.js
import { workerData, parentPort } from "node:worker_threads";
import bcrypt from "bcrypt";
parentPort.postMessage(bcrypt.hashSync(workerData, 12)); // CPU work, off the main loop

Two facts shape everything. First, memory is isolated. A worker does not share variables with the parent; every postMessage payload is structured-cloned, which costs CPU and memory proportional to size — sending a 50 MB buffer this way copies 50 MB. For large binary data you have two outs: transfer an ArrayBuffer in the message’s transfer list (zero-copy — ownership moves, the sender can no longer touch it), or share it for real with a SharedArrayBuffer plus Atomics for coordination, so multiple threads read and write the same bytes with no clone at all.

const ab = new ArrayBuffer(64 * 1024 * 1024);
worker.postMessage({ ab }, [ab]); // transfer list → zero-copy; ab is now detached here

Second, spawning a worker costs real money — a new thread, a new V8 isolate, megabytes of RAM and tens of milliseconds of startup. Spawning one per request is an anti-pattern that often costs more than the work it offloads. The correct shape is a pool: a fixed set of long-lived workers, sized roughly to the number of CPU cores, that pull jobs off a queue. Libraries like piscina implement this; the rule of thumb is pool size ≈ os.availableParallelism(), because more threads than cores just thrash the scheduler.

cluster: many processes sharing one socket

cluster solves a different problem: scaling a network server across all cores. A primary process forks N worker processes, and they all share a single listening socket — incoming connections are distributed across the workers. On non-Windows platforms the default scheme is round-robin in the primary; otherwise the OS distributes via SO_REUSEPORT. Each worker is a full Node process with its own memory and its own event loop, so eight workers genuinely use eight cores for request handling.

import cluster from "node:cluster";
import { availableParallelism } from "node:os";
import http from "node:http";

if (cluster.isPrimary) {
  for (let i = 0; i < availableParallelism(); i++) cluster.fork();
  cluster.on("exit", (worker) => cluster.fork()); // replace a dead worker
} else {
  http.createServer((req, res) => res.end("handled by " + process.pid)).listen(3000);
}

The defining constraint: processes do not share memory. An in-memory session map, a local rate-limit counter, or a process-level cache exists per worker — a request handled by worker A can’t see state worker B set. The fix is to move shared state to an external store (Redis, a database) and treat each worker as stateless. In modern deployments cluster is often superseded by running N container replicas behind a load balancer (or a process manager like PM2), which gives the same parallelism plus restart, rollout, and cross-host scaling — the same “stateless processes, shared store” discipline applies either way.

Choosing, and the failure modes

The decision tree is short. Is the work I/O-bound? Do nothing — it’s already async, and adding threads or processes just adds overhead. Is it CPU work that needs shared or large in-process data? worker_threads pool. Is it scaling a stateless network server across cores? cluster or replicas.

// Pooled, not per-request — piscina sizes ≈ cores and reuses workers
import Piscina from "piscina";
const pool = new Piscina({ filename: "./resize-worker.js" });
app.post("/thumb", async (req, res) => {
  const out = await pool.run(req.body); // CPU work runs off the main loop, on a reused thread
  res.end(out);
});

Two failure modes recur. First: synchronous CPU work on the main thread. This is the hook — a 30%-idle box with a 6-second p99, because one thread is doing all the heavy lifting while seven cores idle. Symptom: event-loop lag spikes, timeouts and failed health checks under load that disappear at low traffic. Fix: offload the CPU work to a worker pool. Second: forgetting cluster workers don’t share memory. You ship in-memory sessions or a local cache, it works in dev (one process), and in production with eight workers a user’s session “randomly” vanishes one request in eight because a different worker handled it. Fix: move that state to a shared store and keep workers stateless.