ASGI and FastAPI: the scope/receive/send contract and the one blocking call that stalls a worker
ASGI hands your app a scope dict plus receive/send awaitables; uvicorn runs one event loop per worker. async def runs on that loop — a single blocking call stalls every in-flight request; plain def is dispatched to a ~40-thread anyio pool. Worker math and honest req/s included.
Launch Friday: the payments team shipped a tiny feature — an async def endpoint that asked the partner’s status API one question, using requests.get. Under launch traffic the partner’s p99 drifted from 180 ms to 6 s, and within a minute the whole service froze in lockstep: checkout, catalog, even /health. One uvicorn worker is one process with one event loop, and every coroutine on it was now waiting behind a single synchronous socket. Kubernetes liveness probes timed out, restarted pods that were not broken, and the restart storm flushed the warm connection pools for good measure. py-spy dump told the story in one line: the only thread, parked in socket.recv. The fix that shipped at 2 a.m. was deleting one keyword — making the handler plain def so FastAPI would push the call into its threadpool — and the postmortem asked exactly the right question: what does async actually promise the event loop?
By the end of this lesson you will know exactly why one blocking call inside async def freezes an entire worker — and how to route each handler to the right execution context before it becomes a 2 a.m. postmortem.
The protocol under the framework
When you see an async def app(scope, receive, send) buried at the bottom of a framework, that is ASGI — the contract between the server (uvicorn, hypercorn) and your application. An ASGI app is one async callable with three arguments: a scope dict describing the connection, and two awaitables — receive to pull events from the server, send to push events back. FastAPI, Starlette, Django — at the bottom they are all this function:
async def app(scope, receive, send):
if scope["type"] == "lifespan": # once per process: startup/shutdown
while True:
event = await receive()
if event["type"] == "lifespan.startup":
await send({"type": "lifespan.startup.complete"})
elif event["type"] == "lifespan.shutdown":
await send({"type": "lifespan.shutdown.complete"})
return
elif scope["type"] == "http": # one scope per request
await receive() # {'type': 'http.request', 'body': b'...'}
await send({"type": "http.response.start", "status": 200,
"headers": [(b"content-type", b"application/json")]})
await send({"type": "http.response.body", "body": b'{"ok": true}'})Three scope types carry the whole web: lifespan (one per process — where startup and shutdown live), http (one per request), and websocket (one per connection, with many receive/send events over its lifetime). Everything FastAPI does — routing, validation, dependency injection — is layered on top of this one calling convention, which is why any ASGI middleware composes with any ASGI framework: each layer is just an app that wraps another app.
One loop, cooperative scheduling, and the stall
uvicorn runs one event loop per worker process. Every async def handler is a coroutine scheduled on that loop, and the loop switches between coroutines only at await points — scheduling is cooperative, not preemptive. That is the entire performance model, and the entire failure model. A coroutine that makes a blocking synchronous call — requests.get, time.sleep, a heavyweight pandas crunch, a sync DB driver — never reaches an await, so the loop never gets the thread back. It is not “that endpoint gets slow.” It is: every in-flight request on the worker, every endpoint, every health check, stops until the call returns. A 200 ms blocking call in a hot handler caps the whole worker at ~5 req/s; a 6-second partner timeout serializes the worker into uselessness, which is exactly the Hook incident.
The dispatch rule: async def on the loop, def in the threadpool
FastAPI decides where your handler runs from its signature alone — there is no runtime detection of blocking code:
@app.get("/right-async")
async def right_async(): # coroutine on the event loop
async with httpx.AsyncClient() as client: # async lib: awaits yield the loop
r = await client.get(PARTNER_URL)
return r.json()
@app.get("/right-def")
def right_def(): # anyio threadpool (~40 threads)
return legacy_soap_client.fetch() # blocking lib, loop stays free
@app.get("/wrong")
async def wrong(): # the Hook incident in three lines
return requests.get(PARTNER_URL).json() # blocks the loop: ALL requests stallPlain def handlers are dispatched to anyio’s worker threadpool, whose default capacity is 40 tokens shared across the process. That protects the loop from blocking code — and caps blocking concurrency at 40; request number 41 queues for a thread. Choosing wrongly costs you in either direction. async def + a blocking call is the catastrophic direction: one handler stalls the worker. def + work that never blocks is the wasteful direction: a handler doing 50 µs of pure-CPU dict shuffling pays a thread handoff measured in tens to hundreds of microseconds plus a token from the 40 — pure overhead. The rule of thumb is honest and short: blocking I/O goes in def (or behind run_in_threadpool), async libraries go in async def, and CPU-bound work belongs in neither — it goes to a process pool or a job queue, because in a thread it still fights the GIL.
An async def endpoint calls requests.get against a partner API whose p99 is ~2 s. The service runs one uvicorn worker. What do callers of OTHER endpoints observe?
Workers, threads, and honest numbers
The deployment unit is the worker process: uvicorn --workers 4 forks four processes, each with its own event loop, its own ~40-thread pool, and its own memory. Capacity scales as workers × pods; within one worker, async concurrency is thousands of parked coroutines (a sleeping coroutine costs ~KBs, not a thread), while blocking concurrency is the 40 tokens. The honest throughput numbers: a trivial JSON echo on one uvicorn worker lands in the thousands of req/s — roughly 3–9k on a cloud vCPU with uvloop, machine-dependent. Add one 20 ms database query and the framework stops mattering: a 10-connection pool serving 20 ms queries ceilings at ~500 req/s no matter what you wrote the handler in, because the framework’s own overhead per request is ~100–300 µs — one to two percent of a single DB round-trip. Benchmark the echo, then benchmark with the database, and size workers from the second number.
You fix the Hook endpoint by making it plain def. Now 100 concurrent requests arrive, each blocking ~2 s on the partner call. What happens?
- 01State the ASGI contract precisely: what is the callable, what are the three scope types, and why does this shape make middleware composable?
- 02Explain FastAPI's def vs async def dispatch rule, the failure mode in each wrong direction, and the honest numbers that govern sizing.
Under FastAPI there is one small contract: an async callable receiving a scope dict and two awaitables, receive and send, with lifespan, http, and websocket scope types carrying everything a web service does. uvicorn runs that callable on one event loop per worker process, and the loop is cooperatively scheduled — it changes coroutines only at await, which makes one blocking synchronous call inside an async def handler a worker-wide outage, not a slow endpoint: checkout, catalog, and /health all freeze behind one parked socket.recv, liveness probes time out, and the restart storm finishes the job. FastAPI’s dispatch rule is mechanical: async def runs on the loop, plain def is shoved into anyio’s threadpool with its default 40 tokens — which protects the loop and caps blocking concurrency at 40, where excess requests queue. Choosing wrongly costs in both directions: blocking inside async def stalls the world; def around microsecond CPU work buys a pointless thread handoff. The honest envelope: thousands of echo req/s per worker, ~100–300 µs of framework overhead, and a database pool that becomes the real ceiling the moment a 20 ms query enters the path — so blocking I/O goes in def, async libraries in async def, CPU work in a process pool or queue, and capacity is workers × pods sized from the DB-bound benchmark, not the echo one. Now when you see an async def handler importing requests, you know exactly what is about to happen to the event loop — and you know the fix is one keyword.
Practice
Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.
Apply this
Put this lesson to work on a real build.