Reverse proxy and API gateway
A reverse proxy fronts your servers; an API gateway adds cross-cutting concerns — auth, rate limiting, routing, aggregation, TLS termination. Both centralize control, but both can become the bottleneck and the SPOF. Know when a gateway earns its keep and when it's overkill.
A company split a monolith into twelve microservices, and within a month every team had independently re-implemented the same four things: JWT validation, rate limiting, request logging, and CORS. Four were subtly different, two had auth bugs, and a client now had to know twelve hostnames and twelve auth quirks. The fix was an API gateway — one front door that did auth, rate-limiting, and routing once, so services could go back to being about business logic. Six months later the same gateway was the reason a config typo at 2 a.m. took down all twelve services at once. The gateway had solved the duplication problem and quietly become the single thing that could fail the whole platform. Both facts are true, and the senior skill is holding them at the same time. By the end of this lesson, you’ll know exactly where that tradeoff lives — and when to push it back.
Forward proxy vs reverse proxy: who they front for
The two are mirror images, and the difference is whose side the proxy is on.
A forward proxy sits in front of clients and represents them to the internet. Your corporate egress proxy, a VPN, a scraper’s outbound proxy — the server on the far side sees the proxy’s address, not the client’s. It exists to serve the client: caching, content filtering, anonymity, egress control.
A reverse proxy sits in front of servers and represents them to the internet. Clients connect to the reverse proxy thinking it is the server; it forwards to one of several backends behind it. It exists to serve the server side: it hides the topology, terminates TLS, caches responses, compresses, and load-balances. Every load balancer from the previous lesson is a reverse proxy with a server-selection policy bolted on; nginx, Envoy, and HAProxy are the canonical examples.
The mnemonic: a forward proxy protects and serves the people making requests; a reverse proxy protects and serves the people answering them.
What an API gateway adds on top
A reverse proxy moves requests. An API gateway is a reverse proxy that has taken on application-level cross-cutting concerns — the things every service would otherwise re-implement (the hook). The standard bundle:
- Authentication & authorization — validate the token / API key once at the edge, so each service can trust an authenticated identity instead of re-checking.
- Rate limiting & quotas — throttle per client/key/route, protecting backends from abuse and noisy neighbors.
- Routing — map public paths to internal services (
/orders→ order-service), often versioned (/v2/...). - Aggregation / composition — fan one client call out to several backends and stitch the responses, so a mobile client makes one request instead of six.
- TLS termination — end HTTPS at the edge so internal hops can be cheaper (composition altitude — see the networking track for how the TLS handshake works; here it’s just where it ends).
- Observability & transforms — logging, tracing headers, request/response rewriting, protocol translation (REST ↔ gRPC).
The value is real: cross-cutting policy lives in one place, enforced uniformly, and clients see one coherent API instead of your internal org chart.
▸Why this works
Why centralize these concerns at all, instead of a shared library each service imports? A library still ships in every service’s process, in every service’s language, and upgrading a security fix means redeploying twelve services and hoping none lagged. Putting auth/rate-limiting at the gateway makes the policy a single deployable you can change once and have apply to everything instantly — and it lets you enforce it on services written in languages that never imported your library. The tradeoff is the mirror image: the thing you can change once is also the thing that, changed wrongly, breaks everything once. Centralization converts N duplicated-policy problems into one consistency win and one blast-radius risk.
The gateway as bottleneck and SPOF
Everything the gateway centralizes, it also concentrates. Every request to every service flows through it, so:
- It’s a single point of failure. If the gateway is down or misconfigured, all services behind it are unreachable, regardless of their own health (the hook’s 2 a.m. typo). The mitigation is the load-balancer playbook: run the gateway as a redundant, horizontally-scaled pool behind its own balancer, with health checks — never a single instance.
- It’s a throughput bottleneck. All traffic plus all the per-request work (token parsing, rate-limit lookups, TLS, aggregation) lands on one tier, which now has to scale ahead of every backend combined.
- It’s a latency tax. It adds a hop and per-request CPU to every call. Usually small, but real, and it’s on the critical path of everything.
- It’s a deployment chokepoint. If every team’s routing config lives in the gateway, the gateway team becomes a bottleneck for shipping — the organizational version of the technical SPOF.
The sidecar / service-mesh alternative
The centralized gateway is one shape; the service mesh is the other. Instead of a single central tier, a mesh runs a small proxy (a sidecar, e.g. Envoy) next to every service instance. Cross-cutting concerns for service-to-service (east-west) traffic — mTLS, retries, timeouts, circuit breaking, load balancing, tracing — move into the sidecars, configured centrally but executed locally. There’s no single proxy all internal traffic must funnel through, so the SPOF and bottleneck of a central tier for east-west calls largely dissolve; the cost is operational complexity (a proxy per pod, a control plane to run) and a small per-hop latency from the local proxy.
The two aren’t either/or in practice: a gateway typically handles north-south traffic (clients → platform: auth, public rate limits, the public API surface), while a mesh handles east-west traffic (service → service: mTLS, retries, internal routing). Reach for a mesh when internal traffic between many services is where your reliability and security problems live; reach for a gateway when the problem is presenting one secure, consistent API to the outside.
▸Common mistake
A gateway is often overkill, and adopting one reflexively is a real mistake. If you have one or two services, a single public API, and no team-duplication problem, an API gateway adds a hop, a SPOF, a new thing to operate, and a deployment chokepoint to solve a problem you don’t have — your reverse proxy / load balancer already does TLS and routing. Worse is the god-gateway: teams keep pushing business logic, request transformation, and response stitching into the gateway until it becomes a distributed monolith that every team must coordinate to change, recreating the very coupling microservices were meant to break. Keep the gateway thin (policy and routing, not business logic), and don’t introduce it until duplicated cross-cutting concerns across several services actually hurt.
Your laptop's traffic goes through a company egress server that hides your IP from the sites you visit; your website's traffic comes in through nginx, which the public sees as the site. Which is which?
A 2 a.m. routing-config typo on your single API gateway makes all twelve services behind it unreachable, even though every service is healthy. What's the root cause and the right structural fix?
Instead of funneling all internal service-to-service traffic through one central gateway, a service _______ runs a small proxy (a sidecar) next to every service instance, so cross-cutting east-west concerns like mTLS, retries, and timeouts are configured centrally but executed locally — with no single central tier to bottleneck.
- 01Distinguish a forward proxy from a reverse proxy, and where a load balancer fits.
- 02List what an API gateway adds beyond a plain reverse proxy, and the core tradeoff.
- 03When do you reach for a service mesh instead of (or alongside) a gateway?
A forward proxy fronts clients (represents you outbound); a reverse proxy fronts servers (clients think it’s the server) and handles TLS termination, caching, and load balancing — every load balancer is a reverse proxy with a server-selection policy. An API gateway is a reverse proxy that has absorbed application-level cross-cutting concerns: auth, rate limiting, path routing, request aggregation, TLS termination, and observability, so each service stops re-implementing them (the hook). That centralization is a genuine consistency win — and a genuine risk: the gateway becomes a single point of failure, a throughput bottleneck, a per-request latency tax, and a deployment chokepoint, so it must run as a redundant, horizontally-scaled pool behind its own balancer and stay thin. The service mesh is the alternative for east-west (service-to-service) traffic: a sidecar proxy per instance, configured centrally but executed locally, with no single tier to funnel through. Reach for a gateway for north-south (the public API) and a mesh for east-west; and don’t add either reflexively — a gateway in front of one or two services is overkill, and a god-gateway stuffed with business logic recreates the monolith it was meant to break. Now when you see a 2 a.m. outage that takes out every service at once while all services are individually healthy, you’ll know to look at the shared tier in front of them first.
Practice
Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.