Zero-downtime deploys: graceful shutdown as a fleet property

Every lesson so far has been about one process dying well. But you never deploy one process — you roll a fleet. A rolling deploy walks through your replicas, replacing each old pod with a new one, and the user-visible promise is “zero downtime”: no request fails, no client notices, even though every single instance is being torn down and rebuilt underneath them. Here’s the unsettling part: a perfect per-instance graceful shutdown does not give you that promise on its own. You can have a flawless SIGTERM handler, correct readiness, clean draining, idempotent requeues — and still take an outage on deploy, because the coordination between instances is wrong. Tear down an old pod before its replacement is ready and you’ve shrunk capacity mid-deploy. Tear down too many at once and you brown out. Let the load balancer keep a stale route for a few seconds and you refuse connections. Send every client Connection: close at the same instant and they all reconnect in one synchronized thundering spike that knocks over the very pods you just brought up. Zero-downtime deploy is not the sum of good shutdowns; it’s a property of how the shutdowns are orchestrated across the fleet. This is the unit’s capstone: everything you’ve learned, lifted from one process to many.

Capacity must never dip: new ready before old drains

The first fleet-level invariant is that total serving capacity never drops below demand during the roll. A rolling deploy is governed by two knobs (Kubernetes names them maxUnavailable and maxSurge, but the concept is universal): how many old pods may be down at once, and how many extra new pods may be spun up beyond the desired count. The safe pattern is surge up, then drain: bring a new pod fully to ready — passing its readiness probe, warmed, connected to its datastores — before you start draining the old one it replaces. If you reverse that, terminating the old pod before the new one is serving, you have a window where the fleet is one instance short, and under steady load that missing capacity is dropped or queued requests. maxUnavailable: 0 with a positive maxSurge encodes “never go below full capacity”; it’s slower and costs transient extra instances, but it’s the setting that actually delivers zero-downtime under load.

Deregister before terminate, sized to real propagation

This is the deregistration race from lesson three, but now as a fleet discipline. Every old pod must be removed from the load balancer’s rotation before it stops accepting — and because deregistration is eventually consistent, you must wait out the propagation before the listener closes. At fleet scale the numbers get concrete and they get big. A typical cloud load balancer’s deregistration delay (connection-draining timeout) defaults to 30–60 seconds; for services with long-lived connections or large uploads, teams push it to 600 seconds or more. The sizing rule that survives contact with production: set the drain window to roughly 3× your p99 request duration so all but the most pathological in-flight requests complete, then clamp it with the guardian timeout. Get this wrong across a hundred pods and every deploy sheds a thin, correlated layer of 502s — invisible in a single request, glaring in the aggregate error rate.

The thundering reconnect: jitter the close

The last trap is one only the fleet can show you. When you drain, you send Connection: close so clients reconnect to healthy instances — correct. But if every pod closes its keep-alive connections at the same moment, every client reconnects at the same moment, and a fleet of thousands of clients lands as one synchronized thundering reconnect on the surviving (and newly-started, cold) pods — a self-inflicted thundering herd, the exact failure mode the circuit-breaker and idempotency units kept circling. The fix is the same one those units used: jitter. Spread connection close over a randomized window instead of a single instant, so reconnections smear across time rather than spiking. Combined with surge-up capacity and properly-sized deregistration, jittered close is what turns “every instance shut down correctly” into “the fleet never hiccupped.” Zero downtime is graceful shutdown plus coordination: capacity, ordering, and spread.