awesome-everything RU
↑ Back to the climb

Networking & Protocols

Latency math

Crux Geography, medium, and hops each add measurable delay. Knowing the numbers lets you distinguish bugs from physics — and design around the floor you cannot lower.
Your altitude — climbing toward senior
ZeroJuniorMiddleSenior
You are at middle altitude — in the sky
◷ 12 min

A support ticket says “latency spiked from 30 ms to 500 ms.” Is that the database? The network? The CDN? Before you read a single log, you need the floor numbers — the latency physics forces on every route. Once you know them, anomalies become obvious.

The latency formula

One-way propagation delay = distance / signal speed.

Light in vacuum: 300,000 km/s. Light in glass fibre: ~200,000 km/s (the glass slows it by ~33%). These numbers give you latency floors no software can beat.

Latency floors by route
NYC → London (5,500 km)
28 ms min → 70–90 ms real RTT
NYC → Sydney (16,000 km)
80 ms min → 200–220 ms real RTT
Same continent (2,000 km)
10 ms min → 20–30 ms RTT
LAN (100 m, Cat6)
<0.5 µs propagation
LEO satellite (550 km alt)
~20–50 ms RTT total
GEO satellite (36,000 km alt)
~600 ms RTT total

Per-technology latency

TechnologyTypical RTTBandwidthNotes
GEO satellite~600 ms25–100 MbpsPhysics: 36,000 km altitude
LEO satellite (Starlink)20–50 ms50–300 MbpsMuch closer orbit
DOCSIS cable (loaded)50–200 ms100–500 MbpsBufferbloat under saturation
4G LTE30–60 ms10–100 MbpsScheduling adds to propagation
5G sub-6 GHz15–30 ms100 Mbps–1 GbpsBetter scheduling
FTTH fibre2–10 ms1 Gbps symmetricISP edge to home
Gigabit LAN<1 ms1 GbpsWithin building

Real RTT is always higher than the propagation floor because routing adds distance, each router adds a small processing delay (~1 µs for modern hardware), and queuing can add milliseconds under load.

Why real RTT exceeds the floor

Take NYC → London (theoretical floor: 55 ms RTT at 200,000 km/s). Real RTT is 70–90 ms — 30–60% above the floor. The excess comes from:

  1. Routing overhead: cable routes are not straight lines; the actual cable path is longer than the great-circle distance.
  2. Router processing: each intermediate router reads the IP header and looks up the routing table (~microseconds each, dozens of hops).
  3. Serialisation delay: time to push a full packet (1500 bytes) onto the wire at the link rate. At 1 Gbps: 1500 × 8 / 10⁹ = 12 µs. Negligible for high-bandwidth links, significant at 10 Mbps.
  4. Queueing delay: at any bottleneck link, packets wait behind others. Under load this can add tens of milliseconds — covered in lesson 04.

The Internet’s submarine backbone

~500 submarine cables connect the continents. Key facts:

  • Each cable carries 10–30 Tbps via DWDM (Dense Wavelength-Division Multiplexing): dozens of wavelengths on one fibre pair, each wavelength ~100–400 Gbps.
  • EDFAs (erbium-doped fibre amplifiers) regenerate the optical signal every ~80 km without converting to electrical.
  • Cable failures (anchor strikes, undersea landslides, ship anchors) happen monthly; redundancy and BGP rerouting keep traffic flowing.
  • Hyperscalers (Google, Meta, Microsoft) own private cables — MAREA, Dunant, Curie — to guarantee capacity for their traffic.
Quiz

A developer says 'we just upgraded our NYC→Sydney link to 100 Gbps and latency didn't improve.' Why not?

When a network path behaves unexpectedly, these tools locate the layer:

# Linux: interface counters — errors, drops, overruns
ip -s link show eth0

# NIC settings: speed, duplex, auto-neg
ethtool eth0

# Vendor NIC statistics: CRC errors, link restarts
ethtool -S eth0 | grep -E "rx_crc|rx_error|tx_error"

# Wi-Fi signal and rate
iw dev wlan0 link

# Traceroute with ICMP timestamps (shows per-hop RTT)
traceroute -I 8.8.8.8
mtr --report 8.8.8.8

Interpreting what you see:

  • rx_crc_errors > 0: frames arriving garbled — bad cable, dirty SFP, or marginal signal. Replace cable or transceiver first.
  • Link auto-negotiated to 100 Mbps when you expected 1 Gbps: cable or port issue forced fallback. Replace the cable (Cat5e damaged pair).
  • Traceroute RTT jump at hop N: latency added at that router or the link between hop N-1 and N. Not necessarily the router’s fault — ICMP rate-limiting can make it look slow.
  • Wi-Fi “connected at 54 Mbps”: client is using legacy 802.11g rates — very far from AP, or old device. Move AP or device closer.
Order the steps

Order link technologies from highest to lowest typical real-world round-trip latency:

  1. 1 GEO satellite (~600 ms RTT — orbit is 36,000 km)
  2. 2 DOCSIS cable modem under load with bufferbloat (~100–200 ms RTT)
  3. 3 LEO satellite Starlink (~20–50 ms RTT — orbit only 550 km)
  4. 4 4G LTE (~30–60 ms RTT)
  5. 5 FTTH fibre home connection (~5–10 ms RTT to ISP edge)
  6. 6 Gigabit LAN Ethernet (<1 ms RTT in building)

Propagation delay calculation

1/3
Why this works

Why traceroute lies. Many routers rate-limit ICMP packets used by traceroute — you may see * * * hops or unusually high RTT at a hop even though the path beyond it is fine. Use mtr (Matt’s Traceroute) for a live view that aggregates many probes, or traceroute -T (TCP mode) which router ACLs less often block. Never conclude “the problem is at hop N” just because traceroute shows high RTT there unless everything beyond it is also broken.

Recall before you leave
  1. 01
    Light in glass travels at ~200,000 km/s. NYC → Sydney is ~16,000 km. What is the theoretical one-way propagation delay, and why is the real RTT 200–220 ms rather than 160 ms?
  2. 02
    How does DWDM multiply fibre capacity without needing more fibre?
  3. 03
    rx_crc_errors is non-zero on a 10G NIC. What does this indicate and what do you do?
Recap

Propagation delay = distance ÷ signal speed (~200,000 km/s in glass). The floors: 28 ms one-way transatlantic, 80 ms transpacific, ~4 ms to LEO satellite, ~120 ms to GEO. Real RTT exceeds the floor by 25–60% due to routing geometry, router processing, and queuing. The Internet’s backbone is ~500 submarine cables using DWDM (dozens of wavelengths per fibre) with EDFA amplifiers every 80 km. Key debugging tools: ip -s link show, ethtool -S (CRC errors, link state), mtr (traceroute with aggregated probes). When traceroute shows high RTT at a single hop but the path beyond is healthy, it’s usually ICMP rate-limiting — not a real bottleneck.

Connected lessons
appears again in162
Continue the climb ↑Bufferbloat and congestion
shortcuts expand
search
K
prev piece
k
next piece
j
cycle tier
t
this menu
?
sources4
expand
  1. 01
  2. 02
  3. 03
  4. 04

Trademarks belong to their respective owners. Editorial reference only.