Structured concurrency: TaskGroup, except*, asyncio.timeout, and why bare create_task is a leak

The bare create_task leak

asyncio.create_task returns a Task and walks away — and that orphan has three failure modes worth naming precisely. Disappearance: the event loop keeps only a weak reference to running tasks; if your code drops the returned Task, the garbage collector may collect it mid-execution — the docs themselves instruct you to save a reference. Silent failure: an exception inside a fire-and-forget task is stored on the Task object, waiting for someone to await it; nobody ever does, so it surfaces only when the Task is GC’d, as a Task exception was never retrieved log line minutes or hours later, disconnected from any request, trace ID, or alert. Outliving the app: at shutdown, asyncio.run cancels whatever still runs — half-finished writes, sockets mid-handshake — and the Hook’s audit gap is exactly this in production clothes. The pattern’s name in the structured-concurrency literature is the go-statement problem: any construct that lets work escape its caller’s scope makes resource lifetimes, error propagation, and shutdown unanswerable questions. The structured answer: every task has a parent scope that awaits it, hears its exception, and cancels it when the scope dies.

TaskGroup: a scope that owns its children

If the three failure modes above feel abstract, ask yourself: when a payment write dies mid-INSERT during a deploy, who is responsible for retrying it? With bare create_task the answer is nobody. TaskGroup makes the answer structural.

Python 3.11’s asyncio.TaskGroup is an async context manager that owns every task spawned through it. The contract has three clauses. No child outlives the scope: leaving the async with block awaits all children — you cannot forget. Failure is collective: when any child raises (anything except asyncio.CancelledError), the group cancels all remaining children, waits for them to actually finish, and then re-raises. Errors aggregate: because several children can fail during the same cancellation storm, the group raises an ExceptionGroup containing all child exceptions — not just the first — handled with except* syntax, which matches and extracts subgroups by type. Note what except* changes semantically: multiple except* clauses can all run for one ExceptionGroup, each receiving the subgroup of matching exceptions, and anything unmatched re-propagates. Compare this with the previous lesson’s gather: on failure, gather hands you the first exception while surviving siblings keep running unsupervised — with side effects, open connections, and no one to await them. TaskGroup’s cancel-on-failure is the difference between an error and an error plus a leak.

import asyncio

async def enrich_order(order_id: int) -> dict:
    try:
        async with asyncio.TaskGroup() as tg:
            user  = tg.create_task(fetch_user(order_id))
            items = tg.create_task(fetch_items(order_id))
            fraud = tg.create_task(fraud_score(order_id))
        # here: ALL three finished OK — results are safe to read
        return {"user": user.result(), "items": items.result(),
                "fraud": fraud.result()}
    except* FraudServiceError as eg:        # subgroup of matching errors
        for exc in eg.exceptions:
            log.warning("fraud check failed: %s", exc)
        return await enrich_without_fraud(order_id)
    # any other ExceptionGroup propagates to the caller intact

Timeouts and cancellation: one mechanism underneath

asyncio.timeout (3.11+) replaced the older wait_for with a composable context manager, and understanding it means understanding cancellation itself. Cancellation in asyncio is not a kill switch: task.cancel() arranges for CancelledError to be raised inside the task at its next await point — a task that never awaits is uncancellable (it is also blocking the loop, so it has bigger problems). The exception then unwinds the coroutine’s stack like any other, running finally blocks and async with exits on the way. asyncio.timeout(5) builds on this: it schedules a timer; if the body is still running when it fires, the timer cancels the body, and the context manager — at its own boundary — converts that CancelledError into TimeoutError. The conversion-at-the-edge design is what makes nested timeouts compose: an inner timeout converts only the cancellations it caused; an outer cancellation passes through untouched. Two disciplines keep this machinery trustworthy. Never swallow CancelledError: a bare except Exception is safe (CancelledError inherits from BaseException precisely so it escapes that net), but except BaseException: pass or except asyncio.CancelledError: continue breaks timeouts, TaskGroup cancellation, and server shutdown in one stroke — if you must intercept it to clean up, re-raise. Cleanup must be cancellation-safe: code in finally runs while the task is being cancelled, and if that cleanup awaits (closing an async connection, rolling back a transaction), it can itself receive a second cancellation. For cleanup that must complete, 3.11+ offers the pattern of shielding just the critical await.

async def transfer(src, dst, amount):
    async with asyncio.timeout(5):                 # cancels body at T+5s
        await debit(src, amount)
        # the commit must not be torn by a timeout/cancel landing mid-flight:
        await asyncio.shield(credit_and_commit(dst, amount))
    # timeout fired? CancelledError became TimeoutError HERE, at the boundary

async def handler():
    try:
        await transfer("a", "b", 100)
    except TimeoutError:
        ...  # the transfer either fully committed (shield) or fully debit-rolled-back

Migrating the fire-and-forget

The Hook’s audit bug has a structured rewrite with explicit tradeoffs. Inline it — tg.create_task(audit.write(event)) inside the request’s TaskGroup — and the write is supervised, retried, and visible, at the cost of its latency joining the request (often fine: an INSERT is single-digit ms). If latency genuinely cannot be spent, the structured fire-and-forget is a long-lived worker task owned by the application’s lifespan scope, fed by a queue: handlers do queue.put_nowait(event) (microseconds), the worker drains it inside its own TaskGroup, and shutdown closes the scope, which drains the queue before the loop dies. Either way, every task has an owner, every exception has an address, and shutdown is a property of the structure rather than a prayer. What does not survive review: create_task with a comment promising someone awaits it eventually.