Distributed Systems
CAP in practice: config and scenario reading
Crux Read real configs, query options, and a partition scenario; predict the CP/AP behaviour and consistency guarantee each one actually buys.
Your altitude — climbing toward senior
ZeroJuniorMiddleSenior
You are at senior altitude — in orbitCAP/PACELC choices are not abstract — they live in consistency levels, write concerns, and quorum settings. Read each config, then predict how it actually behaves when a partition hits.
Goal
Practise the move you make in every design review: translate a consistency-level or quorum config into its concrete CP/AP behaviour and the exact guarantee it gives a reader during a partition.
Snippet 1 — Cassandra tunable consistency
-- N = 3 replicas per key (replication factor 3)
CONSISTENCY QUORUM; -- W and R each require 2 of 3 acks
INSERT INTO orders (id, total) VALUES (?, ?);
SELECT total FROM orders WHERE id = ?; Quiz
Completed
With RF=3 and QUORUM (W=2, R=2), what guarantee does W+R > N give, and how does a partition change the system's stance?
Heads-up Cassandra is AP only at low consistency (e.g. ONE). At QUORUM it requires 2 of 3 acks; a partition that strands the coordinator with one replica fails the operation, which is CP behaviour for that request.
Heads-up The overlap from W+R > N is exactly the staleness guarantee: the read quorum must intersect the most recent write quorum, so a fresh value is observed. Durability is a separate concern.
Heads-up Quorum overlap gives strong consistency per key for these reads/writes, not full linearizability across keys or for concurrent updates. Two concurrent QUORUM writes can still race; LWW resolves them.
Snippet 2 — MongoDB write/read concern
db.accounts.updateOne(
{ _id: id },
{ $inc: { balance: -100 } },
{ writeConcern: { w: 1 } } // ack from primary only
);
// elsewhere, a reporting service:
db.accounts.find({ _id: id }).readPreference("secondaryPreferred"); Quiz
Completed
What consistency hazard does this pairing create, and what is the highest-leverage fix?
Heads-up The default is consistent only for primary reads with majority concerns. Here w:1 plus a secondary read explicitly opts out: the secondary can lag, and a w:1 write is not majority-durable and can be lost on failover.
Heads-up w:1 is faster precisely because it skips waiting for replication — that is the source of the correctness risk (stale secondary reads and rollback on failover), not merely a latency knob.
Heads-up Primary reads fix the stale-secondary read, but a w:1 write is still only on the primary and can be rolled back if that primary fails before replicating. You need w:majority for durability under failover.
Snippet 3 — partition scenario log
12:00:01 region=us-east PUT cart/42 -> 200 OK (local write accepted)
12:00:01 region=eu-west PUT cart/42 -> 200 OK (local write accepted)
12:00:02 link us-east<->eu-west: 100% packet loss (partition)
12:03:10 link restored; anti-entropy reconciliation begins
12:03:10 cart/42 has two divergent versions, equal vector-clock siblings Quiz
Completed
Both PUTs returned 200 during the partition. What does that tell you about the store, and what must happen at reconciliation time?
Heads-up A CP store would have rejected one side during the partition to avoid divergence. Returning 200 on both sides is the AP choice, and the resulting siblings need explicit conflict resolution, not silent self-healing.
Heads-up These are concurrent (equal vector-clock) siblings; LWW would discard one user's cart based on clock skew. Concurrent updates need a real merge (union the cart items) or client choice, not a timestamp coin-flip.
Heads-up Both succeeding is the cause of the conflict, not evidence against it. Independent acceptance on both sides of a partition is exactly how divergence arises in an AP system.
Snippet 4 — Raft/etcd quorum config
# 3-node etcd cluster, one node in a remote DC
cluster:
- { name: n1, dc: primary }
- { name: n2, dc: primary }
- { name: n3, dc: remote } # high-latency link to primary DC
election-timeout: 1000ms
heartbeat-interval: 100ms Quiz
Completed
The remote link to n3 frequently spikes above 1000 ms RTT. What failure mode does this configuration invite, and what is the fix?
Heads-up Raft's liveness depends on heartbeats arriving within the election timeout. An RTT above that timeout makes a healthy-but-slow node trigger disruptive re-elections — high latency behaves like a partition.
Heads-up Lowering the timeout below the link's jitter makes false elections more frequent, not fewer. The timeout must sit comfortably above worst-case RTT to avoid flapping.
Heads-up Adding a slow voter worsens it: quorum size grows and the unstable remote node still triggers elections. Keep voters fast and co-located; make the remote node a non-voting learner.
Recap
Every CAP/PACELC decision shows up as a knob: W+R > N buys read-your-writes overlap but turns CP under partition; w:1 plus secondary reads quietly opts out of durability and freshness; two 200s during a partition are an AP signature whose siblings need a real merge, not LWW; and a Raft election timeout below worst-case RTT manufactures logical partitions. Read the config, name the guarantee it actually gives, then check how it degrades when the link breaks.