One loop iteration, dissected: ready queue, selector poll, timer heap, and which awaits actually yield

One iteration, dissected

You already run coroutines and TaskGroups daily (the concurrency unit); this lesson opens the engine. The default loop is a readable cycle around three data structures: a FIFO ready queue of callbacks, a selector (epoll/kqueue) watching file descriptors, and a timer heap (a heapq of when-sorted handles). One iteration of BaseEventLoop._run_once, simplified but faithfully:

# the shape of one iteration, simplified from CPython asyncio
def _run_once(self):
    timeout = 0 if self._ready else self._next_timer_delay()  # None = block forever
    events = self._selector.select(timeout)   # the ONLY place the loop sleeps
    self._process_events(events)              # readiness -> callbacks onto ready
    now = self.time()
    while self._scheduled and self._scheduled[0].when() <= now:
        self._ready.append(heapq.heappop(self._scheduled))    # due timers join queue
    ntodo = len(self._ready)                  # snapshot! callbacks added DURING
    for _ in range(ntodo):                    # the drain wait for NEXT iteration
        self._ready.popleft()._run()

Read the consequences off the code. loop.call_soon(cb) appends to the ready queue — FIFO, fair, runs this or next iteration thanks to the ntodo snapshot (a callback that keeps rescheduling itself cannot starve I/O). call_later/call_at push a TimerHandle onto the heap; cancelled timers are only lazily purged, so a million cancelled timeouts linger until the cleanup threshold trips. And the selector timeout is the elegant part: the loop sleeps exactly until the nearest timer is due — zero ready callbacks and no timers means a fully blocking select, zero CPU burned. There is no tick, no polling interval, no scheduler thread. The loop is exactly this cycle, run on one thread, forever.

How await actually works — and which awaits do not yield

A coroutine is generator machinery wearing new syntax: it suspends and resumes via the same frame mechanics, driven by send() and throw(). A Task owns its coroutine and drives it in steps: each step calls coro.send(None), the coroutine runs until it either finishes or yields a pending future; the Task then attaches a done-callback to that future and returns — and that return is handing control to the loop, because the step itself was just a callback in the ready queue. When the future resolves, its callback does call_soon(step) and the cycle continues.

The subtlety seniors get burned by: await does not mean “yield to the loop”. Awaiting a plain coroutine is a synchronous call one frame deeper. Awaiting an already-completed future returns immediately without suspending. Control reaches the loop only when an await bottoms out in something genuinely pending:

async def fast_path(cache, key):
    if key in cache:
        return cache[key]      # this branch never yields: it is a function call
    val = await fetch(key)     # yields ONLY if fetch suspends on real I/O
    cache[key] = val
    return val

# A 100%-cache-hit storm makes fast_path "async but never yielding":
# each call runs start-to-finish inside one task step, and a caller
# looping over thousands of hits starves every other task meanwhile.

That is the starvation subtlety: an async function with no pending await runs as one uninterruptible callback. A tight loop over such calls holds the loop for its entire duration — await asyncio.sleep(0) exists precisely to insert an explicit yield point into such loops, and lesson 2 will show the same fact deciding where cancellation can land.

The one rule, with its mechanism attached

Every asyncio tutorial says “never block the loop”. The mechanism is now visible: a blocking call — requests.get, time.sleep, that 200 ms RS256 verify — runs inside the drain phase of one iteration. While it runs, the iteration never gets back to selector.select, so no socket readiness is observed; timers (including every asyncio.timeout in flight — lesson 3) sit in the heap unfired; new connections queue in the kernel accept backlog. You saw in the web-services unit that one blocked worker stalls one request; here one blocked coroutine stalls every request in the process — that is the multiplier that turned a 200 ms verify into a process-wide 230 ms p99 floor. The pressure valve is loop.run_in_executor / asyncio.to_thread, which move the blocking call to another thread and give the loop back its iteration — lesson 4 owns the sizing math and the GIL caveats.

The detector ships in the box. Debug mode — PYTHONASYNCIODEBUG=1 or asyncio.run(main(), debug=True) — logs every callback or task step that holds the loop longer than loop.slow_callback_duration (default 0.1 s), with the coroutine name attached: Executing took 0.213 seconds is the Hook incident finding itself in staging. It also tracebacks never-awaited coroutines to their creation point. The overhead is real, so run it in staging load tests, not as a prod default.

uvloop, honestly

uvloop swaps the whole loop implementation for libuv (the C library under Node.js) behind the same API: install the policy, everything else unchanged. Honest numbers: around 2–4x throughput on socket-heavy workloads — proxies, echo-style services, many small frames — shrinking toward nothing as your own handler code dominates the profile. It makes the machinery (polling, callback dispatch, transport reads) faster; it cannot help the Hook incident at all, because the loop was not slow — it was held hostage. If py-spy shows time in your Python, uvloop moves nothing.