Queues, Streams, Eventing
Message ordering: multiple-choice review
Crux Multiple-choice synthesis across the ordering unit — the throughput cost of total order, per-partition FIFO, partition-key choice, producer reordering, and where order silently breaks in production.
Your altitude — climbing toward senior
ZeroJuniorMiddleSenior
You are at senior altitude — in orbitSix questions that cut across the whole unit. Each one mirrors a decision you make in a real incident — not a definition to recite, but a tradeoff to weigh when ordering and throughput pull against each other.
Goal
Confirm you can connect total-order cost, per-partition FIFO, partition-key choice, producer guarantees, and the production failure modes — the synthesis the lesson built toward.
Quiz
Completed
Why is global total order across all messages fundamentally a throughput tax, not just a config flag you can flip on?
Heads-up It is not a buffering cost — it is a concurrency cost. Ordering everything means serializing everything, which caps you at one consumer's throughput no matter how much memory you add.
Heads-up Faster hardware speeds up the single serialization point but cannot parallelize it. The cap is structural — one partition, one consumer — not a hardware bottleneck.
Heads-up They differ precisely in throughput. Partial order parallelizes across keys; total order serializes everything through one point. Durability is an orthogonal concern.
Quiz
Completed
A wallet service emits credit then debit per account. It worked in staging with one consumer; in prod with eight consumers the debit sometimes lands first. What actually changed?
Heads-up In a consumer group each partition is assigned to exactly one consumer; messages are not all read by everyone. The issue is parallelism across partitions, not duplicate reads.
Heads-up Nothing was dropped. Both events were delivered; they were processed out of order because they landed on partitions handled by different consumers running concurrently.
Heads-up The reorder is structural — parallel processing of independent partitions — not a latency artifact. Removing the accidental single-threading is the root cause.
Quiz
Completed
You need per-account ordering at high throughput on Kafka. What is the correct partition-key choice and why?
Heads-up A random key maximizes parallelism but destroys per-account order — one account's events scatter across partitions and race. Even load is worthless if it breaks the ordering you needed.
Heads-up One key routes every message to one partition: a single-threaded pipeline. That guarantees total order but throws away all parallelism to solve a per-account problem.
Heads-up Hashing a timestamp scatters an account's events across partitions by time, not by entity. Partitioning is by key hash, not sort order; the key must be the consistency boundary.
Quiz
Completed
Which Kafka producer configuration can silently reorder two messages on the same partition?
Heads-up This is the safe default since KIP-185: the broker tracks a per-producer, per-partition sequence number and refuses out-of-sequence batches, preserving order even with 5 in-flight requests.
Heads-up One in-flight request means no later batch can overtake a retrying one, so order holds regardless of the idempotence setting.
Heads-up Per-partition order is a core Kafka guarantee. It is broken only by the specific non-idempotent, multiple-in-flight retry case.
Quiz
Completed
A team grows a Kafka topic from 6 to 12 partitions to add throughput and immediately sees per-key ordering corruption. Why, and what is the standard avoidance?
Heads-up Existing partitions are not deleted; their data stays. The corruption is that a key now hashes to a different partition than its already-queued events, so two partitions briefly hold the same key.
Heads-up Per-partition order still holds after the dust settles; the corruption is a transient window during the remap. The durable fix is to avoid frequent repartitioning, not to use fewer partitions forever.
Heads-up Consumers handle many partitions fine. The reorder comes from the key-to-partition remap during the resize, not from consumer capacity.
Quiz
Completed
A profile service consumes user.updated on an at-least-once queue with concurrent consumers; updates for one user occasionally arrive out of order or twice. What is the strongest design?
Heads-up Idempotency stops double-applying the same message, but it does not order two different messages. Two distinct updates for one user can still apply in the wrong order; you also need a version check.
Heads-up Exactly-once addresses duplicates, not cross-message ordering across concurrent consumers. Treating it as an ordering guarantee is exactly the misunderstanding that ships the bug.
Heads-up That guarantees order by serializing unrelated users against each other — capped throughput to solve a per-user problem. Correct only for a genuinely global ledger, which a profile service is not.
Recap
The through-line is one decision tree: total order needs a single serialization point and caps throughput, so you almost always want partial order — per-key FIFO. Choose the partition key as the entity that must stay consistent (accountId, userId). Order then breaks in three predictable places: a non-idempotent producer with multiple in-flight requests, a repartition that remaps keys, and at-least-once redelivery or DLQ replay. The durable design partitions by key and version-guards against the residual disorder.