Networking & Protocols
QUIC internals: code and trace reading
Crux Read real QUIC stream-ID logic, a packet trace, a send loop, and a frame layout, then predict the behaviour and pick the highest-leverage fix.
Your altitude — climbing toward senior
ZeroJuniorMiddleSenior
You are at senior altitude — in orbitQUIC problems are diagnosed in stream-ID logic, packet traces, and the send path. Read each snippet, predict the behaviour, and choose the fix a senior engineer reaches for first.
Goal
Practise reading QUIC at the wire and code level: decode a stream ID, recognise a healthy versus pathological handshake trace, spot the syscall bottleneck on a fast link, and reason about retransmit packet numbering.
Snippet 1 — stream-ID classification
// Classify a QUIC stream ID by initiator and directionality.
func classify(id uint64) (initiator string, bidi bool) {
if id&0x1 == 0 {
initiator = "client"
} else {
initiator = "server"
}
bidi = id&0x2 == 0
return
}
// Caller passes the stream ID seen on the wire:
i, b := classify(3) Quiz
Completed
For stream ID 3, what does classify return, and which real HTTP/3 entity uses this stream class?
Heads-up Client-initiated bidirectional streams are 0, 4, 8, ... (low two bits 00). ID 3 has both low bits set, so it is server-initiated and unidirectional — not a request stream.
Heads-up Bit 1 of ID 3 is set, so the stream is unidirectional, not bidirectional. Bidi server streams end in 01 (1, 5, 9, ...), not 11.
Heads-up Odd IDs are entirely valid: they are server-initiated. The two low bits encode initiator and directionality, so every value 0..N maps to a real stream class.
Snippet 2 — the handshake trace
t=0.000 dcid=a1b2 type=Initial pkt=0 frames=[Crypto[0..120], Padding]
t=0.045 dcid=a1b2 type=Initial pkt=1 frames=[Crypto[0..120], Padding] # same CRYPTO bytes resent
t=0.051 scid=c3d4 type=Initial pkt=0 frames=[Crypto[0..200], Ack[0]]
t=0.052 scid=c3d4 type=Handshake pkt=0 frames=[Crypto[200..360]]
t=0.101 type=1RTT pkt=0 frames=[Stream(0, fin, 4096B)] Quiz
Completed
The client resends the same ClientHello CRYPTO bytes at t=0.045 as a new packet (pkt=1), before the server's Ack arrives at t=0.051. What is happening, and is it a defect?
Heads-up The retransmit is pkt=1, a fresh packet number — QUIC never reuses a packet number on retransmit. That is precisely what removes the TCP Karn ambiguity; reusing 0 would be the bug, and it did not happen here.
Heads-up The server does respond at t=0.051 with Initial+Ack. The retransmit was a single spurious PTO probe ~6 ms before the response landed — healthy on a 50 ms round-trip path, not a failure.
Heads-up Initial and Handshake each have their own packet-number space by design, so loss in one does not shrink the other's window. The trace shows correct per-space numbering.
Snippet 3 — the send loop
// Hot path: flush queued QUIC datagrams to the socket.
for _, dgram := range batch { // batch may hold 40+ datagrams
_, err := conn.WriteTo(dgram, peer) // one sendmsg syscall per datagram
if err != nil {
return err
}
} Quiz
Completed
At 1 Gbps with 1500-byte datagrams this loop pegs a CPU core and goodput collapses, while the same code is fine on a 10 Mbps mobile link. What is the bottleneck and the highest-leverage fix?
Heads-up The dominant cost here is the per-datagram syscall, not allocation. The fix that scales is UDP GSO batching, which collapses ~40 syscalls into a handful; buffer pooling is a minor second-order tweak.
Heads-up peer is already a loop-invariant local; re-passing it costs nothing. The real bottleneck is one syscall per datagram, addressed by GSO segmentation offload.
Heads-up Adding cores delays saturation but the per-byte syscall cost scales with traffic, so the spiral returns under load. GSO removes the syscalls themselves — the durable fix.
Snippet 4 — the migration frame burst
# Server's view after a client packet arrives from a NEW source address:
recv src=203.0.113.50:55555 dcid=a1b2 bytes=1200
send PATH_CHALLENGE token=0x9f3c... # 64-byte random
# server still has stream data queued for the client
send Stream(0, off=8192, 1500B) # queued application data Quiz
Completed
The server received 1200 bytes from the new, unvalidated address and now wants to flush PATH_CHALLENGE plus 1500 bytes of queued stream data. What does the anti-amplification limit allow, and why does it matter?
Heads-up An established Connection ID arriving from a new, unvalidated address is exactly the migration/spoofing case the 3x limit guards. The server must throttle to 3x the received bytes until PATH_RESPONSE confirms the path.
Heads-up PATH_CHALLENGE is small and fits well within the 3x budget; that is how validation bootstraps. The limit caps the volume, it does not forbid the challenge.
Heads-up The 1200-byte Initial padding and the 3x send limit both exist to bound amplification; the 3x cap applies to any send toward an unvalidated address, including a mid-connection migration.
Recap
QUIC is read at the wire: the two low bits of a stream ID decode initiator and directionality; a handshake trace shows spurious PTO probes as new packet numbers (never reused, so Karn’s ambiguity is gone); a per-datagram send loop reveals the syscall bottleneck that UDP GSO collapses; and a migration burst is bounded by the 3x anti-amplification budget until PATH_RESPONSE validates the new address. Diagnose from the trace and the hot path, then apply the structural fix — GSO, not more cores; idempotency, not disabling features.