Numbers every engineer should know

The latency hierarchy

Ask yourself: how long does it actually take to read from RAM versus a disk, or to call a service in another region? If you can’t answer within an order of magnitude, you’ll approve slow designs without realizing it. Here’s the table that fixes that.

Every operation a system performs sits somewhere on a ladder that spans roughly eight orders of magnitude — from a cache hit that’s effectively free to a cross-continent round trip that’s an eternity. Jeff Dean’s canonical “latency numbers every programmer should know” table is the spine of that ladder. Rounded to the magnitudes that matter:

L1 cache reference                   ~1 ns        (0.5 ns)
Branch mispredict                    ~5 ns
L2 cache reference                   ~7 ns
Main memory (RAM) reference          ~100 ns      (~100× slower than L1)
Read 1 MB sequentially from RAM      ~250 µs      (250,000 ns)
SSD random read                      ~16 µs       (~150× slower than RAM)
Round trip within a datacenter       ~0.5 ms      (500,000 ns)
Disk (HDD) seek                      ~10 ms       (~20× a DC round trip)
Read 1 MB sequentially from disk     ~30 ms
Round trip CA → Netherlands → CA     ~150 ms      (cross-region)

The absolute numbers drift with hardware generations — Dean’s original 2010 figures had no SSD line, and modern NVMe is faster than the ~16 µs here — but the ratios are durable and that’s what you memorize. RAM is ~100× slower than L1. SSD is ~100–150× slower than RAM but ~1000× faster than an HDD seek. A same-datacenter network round trip (~0.5 ms) is ~5000× a RAM access. A cross-region round trip (~150 ms) is ~300× a same-datacenter one and is governed by the speed of light, so no hardware upgrade will ever fix it.

How they scale relative to each other

The single most useful mental trick — popularized as the “if a nanosecond were a second” scaling — is to stretch the ladder into human time so the gaps become visceral. Multiply everything by ~10^9:

• L1 cache (~1 ns) → 1 second.
• RAM (~100 ns) → ~2 minutes.
• SSD read (~16 µs) → ~4.5 hours.
• Datacenter round trip (~0.5 ms) → ~6 days.
• Disk seek (~10 ms) → ~4 months.
• Cross-region round trip (~150 ms) → ~5 years.

Now “reads from RAM instead of disk” stops being abstract: it’s the difference between a 2-minute errand and a 4-month expedition. This is why caching exists, why a hot working set in memory is worth so much, and why a single accidental disk seek or cross-region hop on the request path can dominate everything else combined. The lesson from estimation — find which step dominates — is exactly this table applied: the slowest tier on the path sets the floor.

Throughput orders, not just latency

Latency is “how long for one”; throughput is “how many per second,” and it has its own rough ladder worth carrying:

• Sequential RAM bandwidth: tens of GB/s per socket.
• NVMe SSD: ~3–7 GB/s sequential, hundreds of thousands of IOPS.
• HDD: ~100–200 MB/s sequential, but only ~100 random seeks/sec — random I/O is where disks die.
• Network: a 10 Gbps NIC moves ~1.25 GB/s; 25/100 Gbps in modern datacenters.
• A single well-tuned service: low tens of thousands of simple requests/sec per core-bound box is a sane order-of-magnitude starting point.

The pairing that matters: sequential access is one to two orders faster than random on both disks and (to a lesser degree) memory, because sequential reads amortize seek/setup cost and use prefetch. “Make the access pattern sequential” is one of the highest-leverage performance moves precisely because of this ratio.

Using the numbers to reject designs cheaply

The payoff is cheap rejection: killing a bad design in a sentence instead of a sprint. A few patterns the table refutes instantly:

• “We’ll read it from disk per request at 50K QPS.” An HDD does ~100 random seeks/sec; you’d need ~500 disks. The design is dead — cache it or use SSD/RAM.
• “One synchronous cross-region call per request is fine.” It adds ~150 ms to every request, floor. Move the data closer or make the call async.
• “N+1 queries, but they’re fast.” 100 sequential same-datacenter round trips × 0.5 ms = 50 ms of pure network wait before any work. Batch them.
• “Loop over 10M rows in the app instead of in SQL.” If each row is a fetch, that’s 10M round trips; the ratio between in-process and over-the-network work is thousands-fold. Push the work to the data.

Together these patterns have one thing in common: they all involve crossing a slow boundary many times when one crossing (or zero) would do. When you see a design that calls remote resources per-item in a loop, the table gives you the rejection in seconds — no benchmark needed.