open atlas
↑ Back to track
System Design Foundations SD · 04 · 07

Data distribution: config and code reading

Read real replication config, a partition-key choice, a routing function, and a quorum setting, then do the reasoning: spot the lost-write risk, the hot shard, the mod-N reshard trap, and whether a quorum guarantees a fresh read.

SD Senior ◷ 14 min
Level
FoundationsJuniorMiddleSenior

Distribution bugs live in the config and the routing code: a replication mode, a partition key, a modulus, a quorum number. Read each snippet, reason through the failure it implies, and pick the answer a senior engineer would commit to in review.

Practise the loop you run during a design or incident review: locate the distribution decision in the config or code, trace its failure mode, and choose the change the mechanism actually supports.

Snippet 1 — the replication config

# database replication
replication:
  mode: async          # leader acks before followers receive the write
  followers: 3
  sync_replicas: 0     # none required to ack
# on leader failure: auto-promote the most-caught-up follower
Quiz

What durability risk does this config carry, and what's the minimal change?

Snippet 2 — the partition key

# events table is sharded; choose the shard key
def shard_for(event):
    return hash(event.created_at_day) % NUM_SHARDS   # shard by day
# hottest query: SELECT ... WHERE tenant_id = ? AND created_at_day = ?
Quiz

What's wrong with sharding events by created_at_day, given the workload?

Snippet 3 — the routing function

# cache client picks a node for each key
def node_for(key, nodes):
    return nodes[hash(key) % len(nodes)]   # nodes is a live, autoscaling list

# autoscaler adds/removes cache nodes based on load
Quiz

The cache fleet autoscales, so len(nodes) changes at runtime. What does this routing cause, and what's the fix?

Snippet 4 — the quorum setting

# leaderless key-value store
N = 5          # replicas per key
W = 2          # nodes that must ack a write
R = 2          # nodes that must answer a read
# product requirement: a read must always see the latest acknowledged write
Quiz

Does W = 2, R = 2 with N = 5 meet the requirement, and if not, what's the minimal correct setting?

Recall before you leave
  1. 01
    How do you read durability risk off a replication config?
  2. 02
    Why is hash(key) % len(nodes) unsafe for an autoscaling node set, and what replaces it?
  3. 03
    Given N, W, R, how do you tell if reads are guaranteed fresh, and what's the minimal fix when they aren't?
Recap

Every distribution decision in this unit is something you can read straight off the config or routing code. A replication config with mode: async and sync_replicas: 0 acks before any follower has the write, so a failover loses acknowledged writes — the fix is one synchronous replica. A partition key chosen against the workload (sharding events by day when the hot query filters by tenant) creates a write hot spot and an unroutable query — re-key to match the access pattern. A routing function of hash(key) % len(nodes) over an autoscaling node set remaps nearly all keys on every scale event — replace it with a consistent-hashing ring so only ~1/N move. And a quorum guarantees a fresh read only when R + W > NW = 2, R = 2, N = 5 (sum 4) does not, while W = 3, R = 3 (sum 6) does. The senior habit is to find the distribution decision in the config, trace its failure mode, and choose the fix the mechanism supports — not the one that hides it.

Something unclear?

Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.

shortcuts expand
search
K
prev piece
k
next piece
j
cycle tier
t
this menu
?
sources3
expand
  1. 01
  2. 02
  3. 03

Trademarks belong to their respective owners. Editorial reference only.