awesome-everything RU
↑ Back to the climb

Databases

Partitioning vs sharding: same word, two different things

Crux Partitioning is single-instance query pruning and retention management; sharding is multi-instance throughput scaling. They are complementary and typically composed in production.
Your altitude — climbing toward senior
ZeroJuniorMiddleSenior
You are at middle altitude — in the sky
◷ 10 min

A team adds PARTITION BY RANGE (created_at) to their events table and calls it “sharding the database.” A month later they wonder why write throughput has not improved. Partitioning and sharding are often conflated — they solve different problems on different scales.

Partitioning: one Postgres, table split for pruning

Declarative table partitioning (introduced in PostgreSQL 10) splits a single table into named sub-tables called partitions, all living on the same Postgres instance.

-- One Postgres, one machine, many partitions
CREATE TABLE events (
  id BIGINT,
  tenant_id INT,
  created_at TIMESTAMPTZ,
  payload JSONB
) PARTITION BY RANGE (created_at);

CREATE TABLE events_2026_01 PARTITION OF events
  FOR VALUES FROM ('2026-01-01') TO ('2026-02-01');
CREATE TABLE events_2026_02 PARTITION OF events
  FOR VALUES FROM ('2026-02-01') TO ('2026-03-01');

Postgres gains two things from this:

  1. Partition pruning: a query with WHERE created_at >= '2026-02-01' only scans the relevant partitions, not the whole table.
  2. Instant retention DROP: deleting a month of data is DROP TABLE events_2026_01 — instantaneous, no long DELETE, no dead-tuple bloat, no massive VACUUM.

The critical limitation: partitioning does not increase throughput beyond one machine. All writes still go to the same Postgres primary; all reads share the same CPU and RAM.

Sharding: many Postgres instances, routed by key

Sharding distributes rows across N physically separate Postgres instances. Each instance holds a subset of the data. Writes and reads for a given shard key go to that shard’s Postgres — parallel, independent IO and CPU.

DimensionPartitioningSharding
Postgres instancesOneMany (N)
Increases write throughput?NoYes (~N×)
Primary benefitPartition pruning, fast DROP for retentionScale past one machine’s ceiling
Cross-table joinsFree (all on one instance)Expensive if cross-shard; free if co-located
Operational multiplierLow (one instance still)N× per-shard operations
Available sincePostgreSQL 10 (declarative)Via Citus extension or application-level routing

How they compose in production

Real production systems typically use both — partition within each shard:

Shard 0: Postgres instance holding tenant_id 1–500
  └── events table: PARTITION BY RANGE (created_at)
        ├── events_2026_01 (Jan rows for tenants 1-500)
        ├── events_2026_02 (Feb rows for tenants 1-500)
        └── ...

Shard 1: Postgres instance holding tenant_id 501–1000
  └── events table: same month partitions
        ├── events_2026_01
        └── ...

The shard key (tenant_id) routes traffic to the right Postgres instance. Within that instance, the time partition means: query WHERE tenant_id = 42 AND created_at >= '2026-02-01' prunes to just the February partition. Retention is DROP PARTITION events_2026_01 per shard.

The senior intuition: partition for what you would do on a single machine anyway (time-series retention, query pruning for large tables); shard only when single-Postgres throughput and storage limits are measured and proven binding.

Why this works

Why does Postgres support up to ~1–2k partitions in OLTP workloads but ~10k in OLAP? The partition metadata is loaded into the planner’s working memory on every query — large numbers of partitions slow plan generation. For OLTP queries that touch one or two partitions, the plan overhead from 10k partitions dominates execution time. OLAP queries that scan many partitions in parallel can amortize the planning cost. The practical limit in OLTP is around 1–2k partitions per table.

Quiz

A team adds PARTITION BY RANGE (created_at) to their largest table. What problem does this solve?

Quiz

In a production Citus cluster, what is the typical composition of partitioning and sharding?

Recall before you leave
  1. 01
    What are the two main benefits of declarative table partitioning in Postgres, and what is its key limitation?
  2. 02
    A B2B SaaS shards orders by tenant_id across 16 Postgres instances. Each instance also partitions orders by created_at month. What does the query WHERE tenant_id = 42 AND created_at >= '2026-02-01' do?
  3. 03
    Why is 'partition for manageability, shard for capacity' the senior heuristic?
Recap

Partitioning (PostgreSQL 10+ declarative PARTITION BY) splits one table into sub-tables on a single Postgres instance — it benefits query pruning and makes retention DROP instantaneous, but does not increase throughput past one machine. Sharding distributes rows across N separate Postgres instances by a shard key, giving roughly N× the throughput and storage capacity at the cost of N× the operational complexity. Production systems use both: shard by tenant_id for cross-machine capacity, and partition by time within each shard for pruning and retention. The senior rule: partition freely when the table is large enough to benefit; shard only after measuring and proving that single-Postgres limits are the real bottleneck.

Connected lessons
appears again in140
Continue the climb ↑Co-location and Citus: the invariant that makes sharding usable
shortcuts expand
search
K
prev piece
k
next piece
j
cycle tier
t
this menu
?
sources3
expand
  1. 01
  2. 02
  3. 03

Trademarks belong to their respective owners. Editorial reference only.