Networking & Protocols
Network security: survive a layered DDoS
Reading about defense in depth is not the same as keeping a service up while it is being attacked. Build a small target, drive it into overload with a realistic L7 attack, and add the unit’s layers one at a time — measuring what each layer actually buys before adding the next.
Turn the unit’s layered model into a reproducible loop: instrument the service, reproduce an attack that bypasses naive per-IP limits, then add rate limiting, WAF tuning, and adaptive concurrency in priority order — verifying each layer with before/after metrics under identical load.
Take a small HTTP service with one expensive, cacheable endpoint and keep p99 latency under target and the origin alive under a distributed cache-busting L7 flood — without simply buying 10x more capacity — proving each defense layer with measurements.
- A before/after table: cache-hit rate, origin in-flight peak, 503 rate, legitimate-request p99, measured under the same load — not estimated.
- Evidence that per-IP rate limiting alone fails against the distributed attack (the gap is shown, not assumed), and that adaptive concurrency closes it.
- The WAF/heuristic rule's measured false-positive rate on legitimate traffic, with a stated decision on whether it runs in block or detect mode and why.
- A one-paragraph write-up: which layer stopped which attack variant, in what order you added them, and why adaptive concurrency was the layer that actually saved the origin.
- Add a SYN-flood dimension: enable tcp_syncookies and show the host survives a spoofed SYN flood that drops legitimate connections with cookies disabled.
- Add an on-call runbook: the five signals, the escalation thresholds (request rate 10x baseline, cache-hit below 80% during a spike, source-IP entropy drop), and the order to pull each lever.
- Add mTLS between the edge and origin so the origin only accepts connections from the edge identity, and show a direct-to-origin attack is rejected at the handshake.
- Model attack economics: estimate the attacker's cost (botnet rental) versus your defense cost with and without a shared edge, and write the one-paragraph argument for why defending alone loses.
This is the loop you run in every real DDoS incident: instrument first, reproduce the attack that bypasses the naive defense, then add layers in priority order — edge absorption, rate limiting, WAF, adaptive concurrency — measuring what each one buys before adding the next. The lesson that sticks is that per-IP rate limiting fails against a distributed cache-busting flood, and the layer that actually saves the origin is adaptive concurrency reacting to real overload. Doing it once on a toy service makes the production version muscle memory.