Cancellation semantics: an injected CancelledError, the bare-except immortality bug, and cleanup that must re-raise

Cancellation is an injected exception, not a flag

task.cancel() does not stop anything. It arranges for the loop to throw CancelledError into the coroutine at its next suspension point — mechanically, the Task resumes its coroutine with coro.throw(CancelledError()) instead of send() (lesson 1’s machinery, weaponized). Two consequences define the whole model. First, code between awaits is uninterruptible: a guarantee — your dict update or two-step state change can never be torn in half by cancellation — and a trap — a 30-second pure-CPU stretch is uncancellable for its full duration, because there is no suspension point for the throw to land on. Second, cancellation is only deliverable at awaits that actually suspend: the lesson-1 distinction returns with teeth. An await on a completed future or a synchronously-chaining coroutine is not a cancellation point. For long CPU sections that must stay responsive to cancel, the honest workaround is a periodic await asyncio.sleep(0) — it inserts a real suspension where the throw can land, at the cost of a loop round-trip per check.

The swallowing bug, precisely

How many codebases have an immortal task lurking in shutdown? More than you’d think — and the mechanism is a single wrong word in an exception clause. Since Python 3.8, CancelledError inherits from BaseException, not Exception — a deliberate breaking change made so that the most common defensive pattern stays safe:

async def worker(q):
    while True:
        job = await q.get()
        try:
            await handle(job)
        except Exception:        # does NOT catch CancelledError (3.8+): good
            log.exception("job failed, continuing")
        # except:                # catches BaseException: cancellation dies here
        # except BaseException:  # same bug, spelled honestly

except Exception lets cancellation fly through — the task unwinds and ends in the cancelled state, exactly as the canceller intended. A bare except (or except BaseException) catches it, and if the handler then continues the loop, the task is immortal: cancel() returns True, the exception is delivered, and nothing dies. That was the Hook bug, and it predates 3.8 in spirit — before 3.8, even except Exception swallowed cancellation, which is why old codebases are minefields. The discipline when you do need to observe cancellation: catch it, clean up, re-raise:

        except asyncio.CancelledError:
            await q.nack(job)    # release the in-flight job (bound this! see below)
            raise                # ALWAYS - the canceller is owed the unwind

Swallowing without re-raising has a second cost beyond immortality: await task then returns normally, task.cancelled() is False, and every piece of framework machinery that checks “did my cancel work” — TaskGroup, timeout scopes — is now lied to.

await in finally: the double-cancel trap

Cleanup that itself awaits is where correct-looking code hangs. A cancelled task runs its finally blocks as the exception unwinds — but an await inside finally is a new suspension point, and a second cancel() (an impatient shutdown loop, a timeout scope expiring around you) lands exactly there, abandoning the rest of the cleanup. Worse, the cleanup await itself can block forever — a nack() to a dead broker — and now the task is unkillable-by-politeness. The discipline: bound every await in a cancellation path. Wrap cleanup in asyncio.timeout(...) (next lesson dissects it), or shield genuinely-must-finish work; never let cleanup wait on the same conditions that caused the cancellation.

finally:
    with contextlib.suppress(TimeoutError):
        async with asyncio.timeout(2):   # cleanup gets 2 s, not forever
            await q.nack(job)

TaskGroup: structured cancellation, and whose cancel it was

You know TaskGroup behavior from the concurrency unit: first child failure cancels the siblings, results arrive as an ExceptionGroup. The deep detail is how it tells internal cancellation (the group cancelling its own children) from external (someone cancelled the whole group from outside): each task keeps a cancellation count — cancel() increments it, uncancel() decrements. When a TaskGroup or timeout scope cancels the body for its own reasons, it later calls uncancel() to consume that request and decide: count back to zero means the cancel was mine, so report the sibling failure (or TimeoutError); still positive means an outer cancel is also in flight and must keep propagating. Honestly: uncancel() is framework machinery — application code almost never calls it — but knowing it exists explains why swallowing a CancelledError inside a TaskGroup or timeout scope corrupts their bookkeeping, and why the re-raise rule is not etiquette but a protocol.