Workers and lifecycle: gunicorn process math, preload and fork safety, graceful shutdown

Worker classes and the count math

Gunicorn is a pre-forking master: it binds the listening socket, forks N workers that all accept() from it, and supervises — restarts crashes, recycles, relays signals. The worker class decides what one worker is. A sync worker handles exactly one request at a time: your concurrency equals your worker count, and a slow upstream call holds a whole process hostage. A uvicorn worker runs an event loop: one process handles hundreds of concurrent in-flight requests, as long as everything awaits (match the class to the app you actually built in python/06 — blocking DB drivers belong on sync workers or thread executors, not on a loop). Newer uvicorn ships the class as the separate uvicorn-worker package; the old uvicorn.workers import path is deprecated but everywhere in older configs.

The count math follows from the GIL: threads inside one process do not add CPU parallelism, so processes are the unit of CPU scale. CPU-bound services: workers ≈ cores — more just adds context-switching. I/O-bound async services: 1-2 × cores — each worker is a loop that already multiplexes thousands of sockets; you add workers to use more cores and to isolate failure, not to add concurrency. The classic 2*cores+1 heuristic is a sync-worker-era rule of thumb for mixed I/O; treat every formula as a starting point you re-measure under your own p99.

preload_app and fork safety

preload_app = True imports the application once in the master before forking. The wins are real: copy-on-write means heavy imports (numpy, pandas, a model) occupy physical memory once across all workers instead of N times, and worker (re)boot is near-instant because the import already happened. The cost is the Hook: everything created at import time exists before the fork and is duplicated into every child — and while Python objects are copy-on-write copies, an open socket is an OS-level file descriptor, and fork() duplicates the descriptor, not the connection. Eight workers inherit one TCP stream; concurrent writes interleave, replies route to whichever process reads first — the corrupted-connection classic, surfacing as users seeing each other’s data or as protocol errors deep in a driver.

The rule: connections are created after the fork. Two idioms:

# gunicorn.conf.py
import multiprocessing
workers = multiprocessing.cpu_count() * 2          # async I/O shape; ~cores if CPU-bound
worker_class = "uvicorn_worker.UvicornWorker"
preload_app = True                                 # CoW savings — with the rule below
max_requests = 5_000
max_requests_jitter = 500                          # de-synchronize recycles
graceful_timeout = 25                              # < terminationGracePeriodSeconds (30)
timeout = 30

def post_fork(server, worker):
    from app import state
    state.init_clients()                           # pools/clients born HERE, per worker

The alternative is lazy init — the client is constructed on first use inside a worker. Either works; what never works is module-level Redis() or a connected pool at import time with preload on. The same hazard applies to random seeds (fork copies the state — identical “random” sequences per worker) and to held locks.

Lifecycle: recycling and timeouts

max_requests recycles a worker after N requests; max_requests_jitter randomizes N per worker so eight workers do not all restart in the same minute and dip your capacity in sync. Be honest about what this is: a mitigation for memory growth, not a fix — it caps RSS by resetting the process while you find the actual leak with tracemalloc (next lesson dissects one). Running it forever as “the fix” means every leak you ship next is invisible.

timeout is the master’s heartbeat contract: a sync worker that stops notifying the arbiter for timeout seconds is presumed stuck (deadlocked, hung on an unbounded call) and is killed and replaced — brutal but correct for one-request-at-a-time workers. On an async worker the same symptom means something different and worse: the heartbeat misses because the event loop itself is blocked — some sync call crept in (the python/11 bridging lesson) — and the kill takes down not one request but every coroutine in flight on that worker. Same timeout, different blast radius; the diagnosis differs too (py-spy, next lesson).

Graceful shutdown and the scaling signals

The shutdown sequence is a contract spanning two systems. Kubernetes marks the pod terminating: the endpoint leaves the Service (readiness-off-first — new traffic stops routing; a small preStop sleep covers propagation lag), then SIGTERM goes to PID 1. Gunicorn’s master stops accepting, signals workers to finish in-flight requests, and waits graceful_timeout before force-killing stragglers; the pod must exit before terminationGracePeriodSeconds or the kernel SIGKILLs mid-drain. So the inequality is: preStop + graceful_timeout safely under the grace period — the same drain pattern you wired in Go services (go/12), because the contract is the platform’s, not the language’s. Scale signals, honestly: CPU per worker tells you compute headroom; queue/backlog depth and p99 latency tell you saturation before CPU does on I/O-bound services. Scale pods, not just worker counts — pods buy isolation, scheduler bin-packing, and restart blast-radius control that fatter pods do not.