Defense-in-depth architecture and attack economics

You deployed a CDN, rate limiting, and a WAF. Then the attacker switches to cache-miss HTTP floods targeting your most expensive database query, spreading across 10,000 IPs each under your per-IP limit. No single defense catches it. You need to understand how the layers interact and when to escalate to humans.

Defense-in-depth architecture: the full stack. No single defense stops all attacks. The layered approach:

1. Anycast edge scrubbing — every CDN PoP is an active scrubbing center. Attacks are ingested at the nearest PoP rather than concentrated on the origin. Combined capacity across 330+ Cloudflare PoPs exceeds 37 Tbps. A 10 Gbps attack becomes a rounding error spread across many nodes.
2. Stateless L3/L4 rate limits — per-ASN, per-prefix rate limits drop obvious amplification sources and SYN flood sources before TCP state is created.
3. WAF at the edge — detects application-layer patterns (SQLi, XSS, bot fingerprints). Running at PL2 (balanced false-positive/coverage), not PL4 (paranoid) for a public API.
4. Token bucket per IP + per user — stops obvious botnets and authenticated user abuse.
5. Adaptive concurrency limiting at origin — when in-flight requests exceed capacity threshold, reject new requests with fast 503. Service remains stable; users get errors instead of timeouts.
6. Observability and human escalation — when automated defenses fail, on-call engineers add custom rules.

mTLS in service meshes with SPIFFE. Istio or Linkerd deploy sidecar proxies on every pod. The control plane (Istiod) runs a SPIFFE-compatible certificate authority. At startup, each sidecar receives a short-lived certificate (1–24 hours). Certificates rotate via SDS (Service Workload API) push — no sidecar restart needed. Every service-to-service call: (1) mTLS handshake (20–50 ms overhead per new connection on older hardware), (2) encrypted payload, (3) both sides verify certificates. Prevents lateral movement if the pod network is compromised. Cost: cert rotation adds operational complexity and monitoring burden (expired cert = infrastructure incident, not a bug).

Protocol/state-exhaustion in depth. SYN floods: each SYN allocates a half-open connection slot in the server’s backlog. When backlog overflows, the server drops new SYN packets from legitimate clients. SYN cookies encode connection state as a cryptographic cookie in the ISN — no memory allocated; legitimate clients reply with a valid ACK that decodes the cookie. ACK floods: RST rate limiting (limit RSTs per second) and firewall suppression of unmatched ACKs. TCP RST injection (on-path MITM): RFC 5961 challenge-ACK forces the attacker to know the exact sequence number rather than just be in-window.

Rate limiting internals: distributed systems complexity. Token bucket with Redis backing: T = min(C, T + R * delta_time). Distributed: each request atomically INCR key; EXPIRE key window. At 100k req/sec, Redis adds 0.5–1 ms per request = 50–100 ms total added latency. Mitigation: local per-server counter + periodic Redis sync (accepts slight inaccuracy, cuts to microsecond decisions). HyperLogLog for approximate rate limiting: ~1.6 kB per sketch, ~2% error, suitable for ASN-level or IP-range limits.

Adaptive concurrency: load shedding formula. Track in-flight requests Q. Set max_queue threshold. For new requests, survival probability = max(0, 1 - Q / max_queue). Accept the request with that probability. This creates graceful degradation: at 50% overload, 50% of new requests are rejected; at 100% overload, all new requests are rejected. Users see fast 503s instead of queue timeouts (which can be 30+ seconds). Circuit breakers reject requests to backends with recent failure rates above threshold — preventing cascading failure across the entire service graph.

2026-05-15 14:23:00 | requests=45000/sec | score-p50=0.5 | score-p95=3.2 | score-p99=8.1
2026-05-15 14:24:00 | requests=120000/sec | score-p50=6.1 | score-p95=14.2 | score-p99=28.5
2026-05-15 14:25:00 | requests=450000/sec | score-p50=12.3 | score-p95=19.8 | score-p99=32.1
2026-05-15 14:26:00 | blocked=380000 | allowed=70000 | score-threshold=5

Attack economics. Attacker cost: ~$50–500/month for a botnet service capable of 10 Gbps sustained. Origin defense cost: ~$500/month CDN bill for 10 Gbps sustained traffic. If the attacker can generate 100 Gbps, the origin cannot match. But a CDN with global PoPs ingests 100s of Tbps and spreads the cost across millions of customers — per-customer cost is tiny. The economics favor the attacker only if you defend alone. Sharing infrastructure (CDN) is the answer: you cannot out-scale a botnet; you can make the attack economically unattractive by making it fail.

Observability at attack time. Key signals: (1) request rate per second — 10x normal is suspicious; (2) geographic distribution of sources — all from one ASN is suspicious; (3) anomaly score p99 — normal users score <1, attack traffic scores >10; (4) cache-hit rate — attack traffic targeting unique parameters shows sudden drop. Alert thresholds: request rate 10x baseline, anomaly score p99 spike, source-IP entropy drop (100 IPs instead of 100,000), or cache-hit rate drop below 80% during a traffic spike. Human escalation if attack persists >5 minutes or exceeds 50 Gbps.