Load balancing L4 vs L7: what each layer can and can''''t see

A team splits a monolith: /api/* should go to the new service, everything else to the old one. They already run an AWS NLB for “performance”, so they add a rule to route by path — and discover there is no such rule. The NLB cannot see the path. It forwards raw TCP; the URL only exists once HTTP is parsed, and an L4 balancer never parses HTTP. The migration stalls until someone swaps in an ALB. The lesson cost a day: the layer you pick decides which routing is even possible.

L4 forwards bytes; L7 reads requests

A Layer-4 balancer operates at the transport layer. It sees the TCP/UDP connection — source IP, destination IP, ports — and forwards packets to a backend without ever decoding what is inside. It is essentially a fast, connection-aware NAT. Because it never parses the payload, it is protocol-agnostic (it balances any TCP or UDP service, not just HTTP) and extremely cheap per packet — AWS’s NLB handles millions of requests per second at microsecond-level per-packet overhead and hands out static IPs.

A Layer-7 balancer operates at the application layer. It terminates the client connection, reads the full HTTP request — method, host, path, headers, cookies — and then opens its own connection to a chosen backend. That terminate-and-re-originate is the whole source of its power and its cost. Once the balancer holds the parsed request, it can route on /api/* vs /, on Host: header, on a cookie, or on a custom header for canaries. The price: it must buffer and parse every request, and if TLS is in play it decrypts and re-encrypts — real CPU per request, and it only speaks the protocols it understands (HTTP/1.1, HTTP/2, gRPC).

The decision is not “which is better” but “what do I need to see.” If you only need to spread raw TCP across identical backends, L4 is faster and simpler. The moment routing depends on anything inside the request, you need L7 — and no amount of L4 tuning will conjure a URL it never decoded.

Where TLS terminates changes everything

TLS termination is the cleanest lens on the L4/L7 split. An L7 balancer ends the encrypted session at the edge: it holds the certificate, decrypts the request, reads the plaintext HTTP, makes its routing decision, and forwards — either as plaintext inside the trusted network or re-encrypted to the backend. That decryption is exactly what lets it route on path or header. It also centralises certificate management and offloads crypto from your app servers.

An L4 balancer normally cannot terminate TLS, because terminating means decrypting, and decrypting means becoming HTTP-aware. So it does TLS passthrough: the encrypted bytes flow straight to the backend, which holds the cert and terminates. That keeps the balancer blind by design — great for end-to-end encryption and for non-HTTP protocols, useless if you wanted path routing. This is the concrete reason “just use the fast one” backfires: the fast one literally cannot read what you need to route on.

Algorithms, health checks, and pulling dead backends

Both layers still have to choose a backend. Round-robin spreads connections evenly and is fine when requests cost roughly the same. Least-connections sends the next request to the backend with the fewest active connections — far better when request durations vary widely, because one slow endpoint won’t keep getting piled on. Hashing (by client IP, or consistent hashing) pins a given key to a given backend, which is how an L4 balancer fakes stickiness without cookies.

Health checks are what keep the pool honest. Active checks probe each backend on an interval (say every few seconds) and pull it from rotation after N consecutive failures; passive checks mark a backend down after it returns errors to real traffic. Open-source nginx leans on passive checks; ALB/NLB and nginx Plus do active probing. The failure to internalise: a backend that crashes is only removed after the health check notices — so a too-slow interval means a window where live traffic keeps hitting a dead box and 502s leak to users.

Sticky sessions and connection draining: two ways deploys break

Sticky sessions (session affinity) pin a client to one backend — via an L4 IP hash or an L7 cookie — so in-memory session state stays put. It works, and it quietly hurts: load stops distributing evenly (one whale client hammers one box), and that backend can’t be drained cleanly because it owns sessions nobody else has. The senior reflex is to make backends stateless (sessions in Redis or a JWT) so any backend can serve any request; reach for stickiness only when you genuinely can’t externalise state.

The deploy-killer is missing connection draining. When you remove a backend during a rollout, in-flight requests are still being served on it. If the balancer rips it out immediately, those requests die — users see resets mid-checkout. Connection draining (AWS calls it deregistration delay) tells the balancer to stop sending new connections to the backend but let existing ones finish for a grace window. On AWS the default is 300 seconds, tunable from 0 to 3600. Set it shorter than your longest request and you’ll still cut live requests; forget to enable it and every deploy is a small outage. Pair it with health checks: drain, wait, then terminate.