Pool sizing: why bigger is not faster

A team sees the database is the bottleneck, so they raise the connection pool from 20 to 100, expecting roughly five times the throughput. Throughput drops. Latency climbs, CPU on the database spikes, and the work the server actually finishes per second goes down. They raised the one number that looked like a throttle and made everything worse. The pool was not too small — it was now far larger than the number of queries the database hardware can run at once, and every extra connection past that point is not parallelism, it is contention.

More connections is not more parallelism

It feels obvious that more connections means more concurrent work. It does not, and the reason is physical. A database server has a fixed number of CPU cores and a fixed number of disk spindles (or SSD queues). At any instant it can only truly execute as many queries as it has cores; everything else is waiting. A connection that holds a query is not magically running — it is competing for a core to run on.

So when you open 100 connections on an 8-core box, you do not get 100 queries running at once. You get 8 running and 92 fighting over those 8 cores, plus all the overhead of switching between them. The pool size sets how many queries can be in flight; the hardware sets how many can actually progress. When the first number greatly exceeds the second, the gap is pure waste.

The three taxes of an oversized pool

Each connection past the useful count does not sit quietly. It costs you in three ways at once:

• Context switching. The OS time-slices many connections across few cores. Each switch flushes CPU caches and costs scheduler time. With 100 active connections on 8 cores, the box spends a growing share of CPU just changing which query is running instead of running queries.
• Lock and latch contention. Concurrent transactions contend for row locks, and the database’s own internal latches (buffer pool, WAL, lock tables) are shared. More concurrent connections means more contention on those shared structures — and contention is super-linear, so it gets worse faster than the connection count grows.
• Memory. In Postgres each connection is a forked backend process holding ~2–3 MB of baseline memory, and worse, work_mem is allocated per operation per connection. A sort-heavy query at work_mem = 16 MB across 100 connections can reserve gigabytes; the same workload at 10 connections cannot.

The formula: small, derived from hardware

HikariCP’s well-known guidance turns this into a starting formula:

connections = (core_count × 2) + effective_spindle_count

The intuition: while one query waits on disk I/O, another can use the freed CPU core — so you want roughly twice the cores to keep them busy through I/O stalls, plus one per independent disk that can serve a seek in parallel. A 4-core server with one SSD lands at (4 × 2) + 1 = 9 connections. Not 90. The number that maximises throughput is shockingly small, and measured benchmarks back it: a pool of ~10 often beats a pool of 100 on the same hardware, finishing more queries per second at lower latency.

Sizing is finding the saturation point

The formula is a starting point, not a final answer — real workloads have a mix of CPU-bound and I/O-bound queries, and the right number is found by measurement. The method is to raise concurrency while watching throughput and latency: throughput rises, then flattens (the saturation point), then falls as contention takes over. The best pool size sits at or just below that flat top — the smallest pool that reaches peak throughput. Past it you are buying latency with no throughput in return, exactly the queueing-knee behaviour from the throughput lesson, now driven by the pool you chose.