Flow control and congestion control

A fresh TCP connection between London and Sydney has no idea how fast the link is. It starts cautiously, doubling its send rate every round-trip until it hits a limit — then it backs off and tries again. This dance between sender and network is how TCP fills a 10 Gbps pipe or politely shares a 1 Mbps one.

Sliding window and flow control

Once established, every TCP segment carries a window size — the number of bytes the receiver is willing to accept beyond the last acknowledged byte. The sender keeps a sliding window of unacknowledged bytes in flight, bounded by the minimum of the receiver’s advertised window and its own congestion window. As ACKs arrive the window slides forward, allowing new bytes. If the receiver’s application is slow to drain the socket buffer, the window shrinks; if the buffer fills entirely the receiver advertises window=0 and the sender pauses. This is end-to-end flow control: the receiver dictates the pace, the sender obeys.

MSS, window scaling, and SACK

Three options negotiated during the SYN/SYN-ACK exchange significantly affect throughput:

Maximum Segment Size (MSS): the largest TCP payload the stack can handle without IP fragmentation. Typical: 1460 bytes on Ethernet (1500 MTU minus 20 IP + 20 TCP headers), or 1220 bytes on tunnelled networks.

Window Scaling (RFC 7323): the 16-bit window field is multiplied by 2^scale, allowing windows up to 1 GiB. Without this, a 64 KiB window cap on a 100 ms RTT link limits throughput to ~5 Mbps regardless of actual link speed (bandwidth-delay product = 64 KiB / 0.1s ≈ 640 KiB/s). Essential for high-bandwidth long-RTT paths.

Selective Acknowledgements (SACK, RFC 2018): the receiver lists exact ranges of bytes received past a gap, so the sender retransmits only the missing pieces rather than everything after the gap. On a lossy long-RTT path SACK can double effective throughput.

Retransmit timer math (RFC 6298)

If a segment goes un-ACKed within the retransmission timeout (RTO), the sender resends it. RFC 6298 specifies:

SRTT ← (1 − 1/8)·SRTT + (1/8)·new_RTT
RTTVAR ← (1 − 1/4)·RTTVAR + (1/4)·|SRTT − new_RTT|
RTO = SRTT + max(G, 4·RTTVAR)

Where SRTT is the smoothed RTT and RTTVAR is RTT variance (EWMAs). The first RTO defaults to 1s; after a timeout it doubles (exponential backoff) until either the segment arrives or the connection gives up. Karn’s algorithm forbids sampling RTT from retransmitted segments because the responding ACK is ambiguous.

Modern Linux uses RACK-TLP (RFC 8985) for faster loss detection: RACK declares a segment lost when a later segment was ACKed and a reorder-window has expired — no need to wait for the RTO timer. TLP (Tail Loss Probe) re-sends the last unacknowledged segment one RTT after the last send to avoid hanging on the final packet of a flight.

Slow start and congestion avoidance

A fresh connection has no idea how fast the network is. Slow start opens the congestion window exponentially (1, 2, 4, 8, … MSS per RTT) until either a loss occurs or the slow-start threshold (ssthresh) is reached. After that, congestion avoidance grows the window linearly (~1 MSS per RTT for Reno/CUBIC).

On loss, the algorithm differs by variant:

• Reno: halves the window, then linear increase.
• CUBIC (Linux default since 2.6.19): reduces less aggressively then probes back with a cubic curve — faster recovery on high-BDP paths.
• BBR: ignores loss entirely as a congestion signal; uses RTT + delivered-bytes measurements instead (covered in the BBR lesson).