Queues, Streams, Eventing
Outbox pattern: multiple-choice review
Crux Multiple-choice synthesis across the outbox unit — the dual-write gap, the single-transaction fix, polling vs CDC relays, at-least-once delivery, and the operational tax.
Your altitude — climbing toward senior
ZeroJuniorMiddleSenior
You are at senior altitude — in orbitSix questions that cut across the whole unit. Each one mirrors a decision you make in a real incident — not a definition to recite, but a tradeoff to weigh when a write and a publish must both land.
Goal
Confirm you can connect the dual-write gap, the single-transaction fix, the polling-vs-CDC relay choice, the at-least-once guarantee, and the operational tax — the synthesis the lesson built toward.
Quiz
Completed
A handler runs INSERT order, then kafka.publish('OrderPlaced'). The pod is OOM-killed after the commit but before the publish returns. What is the resulting failure, and why can't try/catch save you?
Heads-up Phantom events come from publishing BEFORE the commit and then rolling back. Here the order committed first, so the danger is the opposite: a real order whose event was lost.
Heads-up The publish call never completed and its in-memory state died with the pod. There is no durable record that a publish was owed, so nothing retries it. That durable record is exactly what the outbox adds.
Heads-up The DB transaction committed and is independent of the broker call; a later failure in a separate system cannot un-commit it. The two stores have no shared transaction — that is the root problem.
Quiz
Completed
Why does writing the business row and an outbox row in one local transaction give atomicity 'for free', where 2PC across Postgres and Kafka does not?
Heads-up Append-only-ness isn't what makes it atomic. Atomicity comes from both INSERTs sharing one local transaction against one database; the outbox would be just as atomic if it allowed updates.
Heads-up The relay runs much later in a separate process; it is deliberately NOT in the transaction. Atomicity stops at the two local INSERTs — the publish is a separate, retryable step.
Heads-up 2PC across a DB and Kafka isn't merely harder to code — a blocking 2PC coordinator becomes a worse availability problem than the dual write, and Kafka doesn't offer the XA participation you'd want in production.
Quiz
Completed
A team runs a polling relay at a 1s interval and wants events to land faster. They consider dropping to a 25ms poll. What is the senior read on this?
Heads-up An indexed query still costs a round trip, planning, and lock acquisition on the primary every tick. Forty polls a second across replicas is real, constant load even when the table is empty.
Heads-up Polling latency is bounded by the interval, so a tighter interval does lower latency — the problem is the load it costs, not that it's impossible. CDC is the better tool when you truly need sub-100ms.
Heads-up CDC tails the WAL and emits the moment the INSERT commits, giving single-digit-ms latency with near-zero table load — lower latency than polling, not higher.
Quiz
Completed
Why is the outbox at-least-once rather than exactly-once, and what does that force on every consumer?
Heads-up Idempotent producers dedupe retries within one producer session, not across a relay crash-and-restart that re-reads an unsent row. The outbox guarantee is at-least-once end-to-end; dedupe is the consumer's job.
Heads-up That just moves the gap and converts a recoverable duplicate into an unrecoverable lost event — crash after the UPDATE and before the publish, and the row looks done but nothing was sent. Publish-then-mark is the strictly safer order.
Heads-up At-least-once tolerates duplicates only because consumers dedupe them. Processing OrderPlaced twice must charge the card once; idempotency is the load-bearing half of the pattern, not optional.
Quiz
Completed
You scale the polling relay from one replica to three for throughput and immediately see every event published roughly three times. What is the fix?
Heads-up The outbox scales fine across replicas; you just need row-claim coordination. SKIP LOCKED lets multiple workers share the table without overlap.
Heads-up Offset intervals don't prevent two workers from selecting the same WHERE sent=false rows on overlapping ticks. Only a lock claim makes the batches disjoint.
Heads-up At-least-once tolerates the rare crash-induced duplicate, not deliberately tripling every event; that wastes throughput and hammers consumers. Coordinate the relays with SKIP LOCKED.
Quiz
Completed
The outbox table has grown to several hundred thousand rows, polling queries are slowing, and a nightly DELETE of old sent rows causes lock contention with inserts. What is the better cleanup design?
Heads-up Not marking rows sent makes the relay re-publish everything forever — an infinite duplicate storm, not a cleanup. Sent rows must be reaped, just not with one giant DELETE.
Heads-up More indexes make writes — including the hot INSERT path and the DELETE itself — slower, not faster, and don't address the lock contention. Batch the deletes or partition instead.
Heads-up Coupling cleanup to the business write adds latency and lock pressure to the hot path for no gain. Reaping is background maintenance — batch it or drop partitions out of band.
Recap
Across the unit the through-line is one chain: a dual write across two stores has no crash-safe ordering, so you collapse it into a single local transaction that writes the business row and an outbox row together. A separate relay ships those rows — polling (simple, but interval-bounded latency and constant DB load) or CDC (single-digit-ms, near-zero table load, at operational cost). Because the relay publishes then marks sent in two steps, delivery is at-least-once, so consumers dedupe on a stable event id. The operational tax — table bloat, ordering under parallel relays, competing replicas — is paid with partitioning, batched reaps, aggregate keying, and FOR UPDATE SKIP LOCKED.