CAP & PACELC
CAP is narrow: only during a network partition must you choose consistency or availability. PACELC completes it — else, with no partition, you still trade latency against consistency. Pick per use case along the linearizable-to-eventual spectrum.
A senior engineer says in a design review, “We’re using Cassandra, so we picked AP over CP — we gave up consistency for availability.” It sounds authoritative, and it’s wrong in a way that matters. CAP only forces that choice during a network partition — a rare event. The other 99.9% of the time, when the network is healthy, the real, constant trade-off the team is making is latency versus consistency: every read either pays the cost of coordinating replicas or returns possibly-stale data fast. CAP describes a moment of crisis; the decision that shapes the system every single day is the one CAP doesn’t even mention. PACELC does. Getting these two straight is the difference between cargo-culting a database choice and actually understanding what your data store promises.
CAP, stated precisely
The CAP theorem (Brewer; proved by Gilbert and Lynch) is about three properties of a distributed data store. Before you can use these labels correctly in a design review, you need to know exactly what each one means — and what it does not:
- Consistency (C) — every read sees the most recent write (specifically, linearizability: the system behaves as if there’s one copy of the data and operations happen in a single order). Note this is not the C in ACID — same letter, different idea.
- Availability (A) — every request to a non-failing node gets a non-error response (even if not the latest data).
- Partition tolerance (P) — the system keeps working when the network drops or delays messages between nodes.
The theorem’s real content is narrow and conditional: when a network partition occurs, you must choose between C and A — you cannot have both. If two nodes can’t talk and a write lands on one, the other node either refuses to answer (sacrificing A to stay consistent → CP) or answers with its own possibly-stale data (sacrificing C to stay available → AP). That’s it. The proof is almost trivial once stated this way: a partitioned node that still answers either lies (not C) or stalls (not A).
Why CAP is so often misquoted
Three myths, all worth unlearning:
- “Pick two of three.” You don’t get to drop P. In any real distributed system, network partitions will happen — they’re a fact of physics and operations, not a design choice (the AWS Builders’ Library is blunt that independent failures and partitions are unavoidable). So P is mandatory, and the only live choice is C-vs-A when a partition is in progress. “Pick two” makes it sound like CA is an option; for a distributed store, it isn’t.
- “My database is AP/CP, full stop.” The choice can be per operation, not per system. DynamoDB defaults to eventually-consistent reads (AP-ish) but offers a
ConsistentReadflag for strongly-consistent reads on the same table. Many systems let you pick consistency per request, so labeling the whole database with one letter hides the real, tunable decision. - CAP describes the whole system. It only describes behavior during a partition. The hook’s engineer collapsed “what we do in a crisis” into “what our system is,” and missed the trade-off that’s actually present every day — which is exactly what PACELC adds.
All three myths flow from the same root: treating CAP as a broad system classification rather than a conditional, partition-only statement. When you see a colleague label a database “AP” or “CP” in a design doc, your next question should be “per operation or per system?” and “what happens when the network is healthy?”
PACELC: the rest of the time
PACELC (Abadi) extends CAP to cover normal operation:
PAC | ELC
If Partition → choose Availability or Consistency
Else (normal) → choose Latency or ConsistencyRead it as: if there’s a Partition, trade A vs C (that’s CAP); Else, trade L vs C. The “else” branch is the insight. With the network healthy, a strongly-consistent read must still coordinate across replicas (wait for a quorum, talk to the leader) — which costs latency. To go faster, you read from a nearby replica that might be slightly stale — which costs consistency. So even with zero partitions, you are constantly choosing how much latency to pay for how much freshness. This trade is present on every request, which is why it shapes the system far more than the rare partition does.
Databases get a two-part label: DynamoDB and Cassandra are PA/EL (available under partition, low-latency-but-stale otherwise); a strongly-consistent store like a single-leader SQL database or Spanner leans PC/EC (consistent under partition, consistent-but-higher-latency otherwise). The label finally captures both the crisis behavior and the everyday behavior.
▸Why this works
Why does strong consistency cost latency even when nothing is broken? Because “every read sees the latest write” requires the read to be sure no more-recent write exists anywhere — and the only way to be sure is to coordinate: route through the single leader (lesson 01) or read from a quorum of replicas (R + W > N), both of which add round trips, and cross-region those round trips are the ~150 ms speed-of-light cost from the scalability unit. An eventually-consistent read skips the coordination and answers from the closest copy immediately — fast, but it may miss a write still propagating. So latency-vs-consistency isn’t an implementation wart; it’s the same physics as replication lag, viewed from the read side. PACELC’s “else” branch is just naming the cost of coordination that’s always there, partition or not.
The consistency spectrum, and picking per use case
C-vs-A and L-vs-C aren’t binary; they’re a spectrum of consistency models (Jepsen catalogs them as safety properties — what histories a system may legally produce). From strongest to weakest, the load-bearing rungs:
- Linearizable (strong) — reads always see the latest write; the system acts like one copy. Strongest, costliest, least available under partition.
- Sequential / causal — operations respect a consistent order (causal: effects never appear before their causes — you never see a reply before the message it answers).
- Read-your-writes / monotonic reads — per-client guarantees (you see your own writes; reads don’t go backwards) without global strength — the lesson 01 routing fix lives here.
- Eventual — replicas converge if writes stop, with no ordering or freshness promise in the meantime. Weakest, cheapest, most available.
The senior move is to choose per use case, not per company:
- A bank balance or an inventory decrement that must not oversell → linearizable / CP / EC: correctness beats availability and latency.
- A like count, a feed, a “last seen” timestamp, a product-page view counter → eventual / AP / EL: a few seconds of staleness is invisible to users and buys huge availability and speed.
- A user editing their own profile → read-your-writes is enough — they must see their own change, but global strong consistency is overkill.
This is why the same company runs Postgres for orders and Cassandra/DynamoDB for the activity feed: the data, not dogma, picks the point on the spectrum.
An e-commerce platform runs three data types: (A) inventory counts for 'last 3 in stock' banners, (B) the authoritative stock decrement that prevents overselling when a customer buys, (C) a product view-count shown in the listing. Which consistency model fits each, and what is the right match for (B)?
▸Common mistake
The expensive mistake is applying one consistency level to the whole system. Demanding linearizability everywhere “to be safe” buys you the latency and availability cost of CP/EC on data that never needed it (your view counter doesn’t), making the whole product slower and more fragile under partition for no benefit. The opposite mistake is just as bad: running money or inventory on an eventually-consistent store because it was “web scale,” then discovering you double-sold tickets or let a balance go negative during a partition. Match the consistency model to each piece of data’s actual correctness requirement — and remember it’s often tunable per request (DynamoDB’s ConsistentRead, quorum levels), so you can keep most reads cheap and make only the few that must be strong pay for it. The full machinery that implements strong consistency under partition — quorums, leader election, Raft — is the distributed track’s job; here the skill is choosing the right point, not building it.
A colleague says 'CAP means pick two of C, A, P.' Why is this a misleading way to state it for a real distributed system?
Your network is perfectly healthy — no partitions all quarter. By PACELC, what trade-off is your strongly-consistent database still making on every read, and why?
PACELC reads as: if there is a Partition, trade Availability against Consistency; _______, trade Latency against Consistency — so a healthy network still forces a constant choice between fast-but-stale and slow-but-fresh reads.
- 01State CAP precisely and explain the three common ways it's misquoted.
- 02What does PACELC add to CAP, and why does the 'else' trade-off dominate in practice?
- 03Sketch the consistency spectrum and how you pick a point per use case.
CAP is narrow and conditional: only during a network partition must a distributed store choose between Consistency (linearizability — every read sees the latest write, not ACID’s C) and Availability (every non-failing node answers), giving CP (refuse stale) or AP (serve stale). It’s widely misquoted — “pick two of three” is wrong because partitions are unavoidable so P is mandatory; the label is often per-operation (DynamoDB’s ConsistentRead); and CAP says nothing about normal operation. PACELC completes it: if Partition → A vs C; Else → Latency vs Consistency. The else-branch dominates in practice because every healthy-network read still chooses between coordinating for freshness (latency) and reading a nearby stale copy (speed) — the same physics as replication lag, from the read side. Consistency is a spectrum — linearizable → sequential/causal → read-your-writes/monotonic → eventual (Jepsen’s safety properties; see jepsen.io) — and the senior skill is to pick per use case: strong for money and inventory, eventual for like counts and feeds, read-your-writes for a user’s own edits, often tuned per request. Applying one level everywhere is the expensive mistake in both directions. The machinery that implements strong consistency under partition — quorums, leader election, Raft — is the distributed track’s job; this unit’s job was choosing the right point on the spectrum once you’ve distributed the data. Now when you see a design review claim “we chose AP” or “we chose CP,” you’ll know to ask: during a partition or all the time? — and which consistency level each request actually needs.
Practice
Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.