From Nagle to io_uring: evolution of batching

In 1984, John Nagle watched a single telnet keystroke leave a host as a 41-byte packet: one byte of payload wrapped in a 40-byte TCP/IP header. On the congested ARPANET that was 97.5% waste, and thousands of those tinygrams were melting links. His fix was three lines of TCP logic. Forty years later you tune the exact same lever every time you set linger.ms in Kafka or batch SQEs into one io_uring_enter. The fixed cost changed names; the move never did.

The batching insight is timeless: when a fixed cost per operation dominates the variable cost of the payload — a TCP header, a network round-trip, a syscall context switch, a DB connection acquire — group operations so you pay the fixed cost once instead of N times. Everything in this lesson is that one idea, applied at a different layer of the stack across four decades. The only real design knob is the window: how long to wait, or how full to get, before you flush.

Nagle’s algorithm: the original batching tradeoff

RFC 896 (1984) introduced what we now call Nagle’s algorithm. The rule is small: while there is unacknowledged data already in flight, hold back any new small segment and coalesce it with later writes; flush immediately only when an ACK clears the in-flight data, or when you have a full MSS worth of bytes to send. The motivation was brutal arithmetic — a 1-byte telnet keystroke became a 41-byte packet, so 40 of every 41 bytes on the wire were header. Nagle’s rule turned a burst of keystrokes into one packet per round-trip instead of one packet per key.

The cost lands on interactive and request/response traffic. If your application does a small write() and then waits for a reply, Nagle may sit on that last small segment, hoping for more data that never comes — so it waits for the ACK instead, adding up to a full round-trip of dead time. The escape hatch is the TCP_NODELAY socket option, which disables the algorithm so every write goes out immediately. This is why HTTP/2, gRPC, Redis clients, and basically every modern RPC stack set TCP_NODELAY at connect time and do their own batching at the application layer, where they actually know message boundaries.

The Nagle + delayed-ACK deadlock (the postmortem beat)

The famous failure mode is not Nagle alone — it is Nagle interacting with TCP delayed ACK. Delayed ACK is the receiver-side mirror of Nagle: instead of ACKing every segment, the receiver waits (Linux default: up to ~40ms) hoping to piggyback the ACK onto a reply or batch it with the next one. Now compose the two. The sender writes a response slightly larger than one MSS: the first full segment goes out, but the small trailing segment is held by Nagle because the first is still unacknowledged. The receiver got the first segment but holds its ACK under delayed-ACK, waiting to piggyback. Neither side will move. The deadlock breaks only when the 40ms delayed-ACK timer fires.

The symptom in production is unmistakable and infuriating: a protocol that should do thousands of transactions per second mysteriously caps near 25/sec, with latency histograms spiking at a suspiciously round 40ms (or 200ms on some stacks). Marc Brooker’s line — “It’s always TCP_NODELAY. Every damn time.” — is folklore for a reason. The fix is one socket option; the diagnosis is the hard part, because the 40ms is paid by nobody’s CPU and shows up only as wall-clock stall.

The latency-throughput Pareto frontier

Strip away the layer-specific details and every batching system traces the same curve. On one axis: batch window (time or size). On the other two: throughput and per-item latency, which move in opposition.

The senior workflow is not “pick a number” — it is: define the latency SLO ceiling first (say p99 < 50ms), then find the largest batch window that still fits under it, because that window gives you the most throughput you can buy without breaking the contract. Static systems tune this knob once and live with it. Adaptive systems track the curve at runtime: under light load they shrink the window toward zero (latency matters, there is nothing to batch anyway); under heavy load they let batches fill (throughput matters, and items are arriving fast enough that the wait is cheap). Kafka 4.0 quietly encoded this wisdom: the producer linger.ms default moved from 0 to 5ms, because the efficiency win from fuller batches usually pays for the 5ms wait — frequently yielding lower end-to-end latency, not higher, by reducing per-request overhead.

Batch coalescing and request deduplication

There is a sharper variant for cache/lookup workloads where many callers want the same result, not just any throughput. When concurrent requests miss the cache for the same key, you can collapse them into a single inflight load instead of N duplicate loads. Worker A misses key K and starts the DB query; worker B (and C, and D…) also miss K, see A’s request already pending, and attach to it rather than firing their own. One result fans out to all of them.

This is the cure for the cache stampede / thundering herd: a hot key expires, 100 requests arrive in the same millisecond, and without coalescing all 100 hammer the database at once — often enough to knock it over right when traffic is highest. With coalescing, those 100 misses become 1 query and 99 free-riders. The implementations are everywhere under different names: Go’s golang.org/x/sync/singleflight, Java’s Caffeine AsyncLoadingCache, request collapsing in Varnish and most CDNs. GraphQL’s DataLoader takes it one step further by combining coalescing with windowed batch loading: every distinct key requested within one tick is deduped and the unique keys are bundled into a single batched backend query, which is also how DataLoader kills the N+1 query problem.

Why the lineage matters

The whole point of seeing Nagle, Kafka, and io_uring as one family is that the tuning method transfers. Each is just a different fixed cost wrapped in the same lever, so the diagnostic question is identical every time.

io_uring is the cleanest modern echo: instead of one syscall per I/O, you fill a ring of submission queue entries (SQEs) in shared memory and submit a whole batch with a single io_uring_enter — amortizing the kernel-boundary crossing across many operations, exactly as Nagle amortized the header across many keystrokes. Same lever, new fixed cost. Once you see the pattern, the work is always the same three steps: measure the fixed cost vs. the variable cost, confirm the fixed cost actually dominates, then size the window to the largest value your latency SLO allows.