Sockets in production: timeouts, limits, and fd exhaustion

Idle sockets need a timeout or they pin a file descriptor forever

The 3 a.m. incident in the hook is reproducible in any Node server that ships without three deliberate defenses. Here is how each failure mode works and exactly what stops it.

A TCP socket, once accepted, occupies a file descriptor — a finite, per-process kernel resource (often defaulting to 1,024 or capped by ulimit -n). Node does not impose an idle timeout by default: a socket that connects and then goes silent will sit open indefinitely. A misbehaving or dead client — one that crashed without a clean close, or a half-open connection where the FIN never arrived — therefore leaks a descriptor that is never reclaimed. Accumulate enough and the process can no longer accept() anything: the hang in the hook.

The defense is socket.setTimeout(ms), which fires a 'timeout' event after ms of inactivity (no reads or writes). Crucially, the timeout event does not close the socket for you — it just notifies you — so you must act:

server.on("connection", (socket) => {
  socket.setTimeout(30_000);                 // 30s idle budget
  socket.on("timeout", () => {
    socket.destroy(new Error("idle timeout")); // YOU must close it
  });
  socket.on("error", () => {});               // swallow post-destroy errors
});

For HTTP servers, the same idea is exposed as server.setTimeout(), plus server.headersTimeout, server.requestTimeout, and server.keepAliveTimeout — the last guards the gap between keep-alive requests. The Slowloris attack is precisely the weaponized version of the hook: an attacker opens many connections and dribbles bytes slowly to keep each just under any naive timeout, exhausting your connection capacity with almost no bandwidth. Header and request timeouts are what stop it.

Keep-alive: reuse the handshake, but bound the idle

Keep-alive is the reuse of one TCP connection for many requests, avoiding a fresh handshake (and TLS negotiation) each time — a large latency win. But an idle kept-alive connection still holds a file descriptor and a slot. The tuning tension is real: too short a keepAliveTimeout and you throw away the reuse benefit, reconnecting constantly; too long and idle clients hoard descriptors you could serve to active ones.

There is also a notorious race between a server closing an idle keep-alive connection and a client sending one more request on it: the request lands on a socket the server is tearing down, producing an ECONNRESET the client sees as a failed request. The mitigation is to make the server’s keepAliveTimeout longer than the upstream load balancer’s idle timeout, so the LB always closes first, on a connection it knows is idle — a classic source of intermittent 502s when reversed.

ECONNRESET and EPIPE are not bugs — they are Tuesday

Two error codes dominate production socket logs, and treating them as exceptional is a mistake.

Both are emitted as 'error' events on the socket, and the iron rule from the TCP lesson applies with force in production: an unhandled socket 'error' throws and takes down the entire process, killing every other in-flight connection with it. One misbehaving client must never be able to crash a server serving thousands. So every socket — accepted server-side and opened client-side — needs an 'error' handler that logs-and-cleans-up rather than rethrows. The asymmetry to internalize: ECONNRESET is something done to you (the peer reset), EPIPE is you writing into a closed pipe — but both resolve to “this one connection is dead, release it, keep serving everyone else.”

Bound connections, then drain gracefully

The hook’s root cause was unbounded resource growth. server.maxConnections caps concurrent sockets: past the limit, the server stops accepting (the OS backlog absorbs a few, then refuses), which sheds load instead of collapsing into fd exhaustion. It is a blunt instrument but a real backstop. Pair it with an OS-level ulimit -n raised to a deliberate number and you have an upper bound you chose rather than one you discover at 3 a.m.

The other half is graceful shutdown. On SIGTERM (a deploy, a scale-down), you must drain rather than drop: server.close() stops accepting new connections and fires its callback once all existing ones finish — but it will wait forever on a long-lived idle keep-alive socket, so you also iterate active sockets, signal them to finish, and destroy() any stragglers after a deadline. Skipping this turns every deploy into a wave of ECONNRESETs for in-flight users.