OAuth in production: audience attacks, observability, and real failures
Audience confusion attacks, token storage by client type, the observability metrics that detect compromise, and the four real-world OAuth failures that cost millions.
A JWT validator logs a WARN when the aud claim does not contain the expected value, then proceeds to accept the token anyway because “aud contains the caller’s client_id.” The token was issued for a different application. The validator just accepted a token intended for someone else — audience bypass, a real CVE pattern seen at Microsoft Azure AD in November 2024. By the end of this lesson you’ll know how to write an aud check that cannot be inverted, where each client type must store its tokens, and which six metrics tell you your OAuth deployment is under attack before users start calling support.
Audience attacks
When one authorization server issues tokens for multiple resource servers (API-A and API-B), a validator that accepts any valid aud can be exploited: an attacker who obtains a token for API-A presents it to API-B.
The correct aud check: does the token’s aud claim contain this resource server’s hard-coded identifier? Not “does aud contain any known client_id?” — a logical inversion that bypasses the protection.
RFC 8707 — Resource Indicators: The client specifies a resource parameter in the /token request, naming the target resource server. The IdP mints a token with that audience only. If the client needs tokens for both API-A and API-B, it calls /token twice with different resource values. Inconvenient but correct.
Token Exchange (RFC 8693): A resource server that needs to call a downstream service can present its incoming token and request a new token valid at the downstream, with a narrowed audience. This is “least privilege” for service-to-service hops.
The cnf (confirmation) claim also narrows binding: DPoP sets cnf.jkt to the public key thumbprint. A token with cnf can only be used with proof signed by the corresponding private key — audience binding plus sender binding together.
if token.aud.includes(EXPECTED) { accept }
Token storage by client type
Once you’ve locked down audience validation, the next question is: where do you actually keep those tokens so XSS or a compromised dependency can’t read them? The answer differs by client type. Where tokens live determines their exposure surface:
Browser SPA:
- Access token: JavaScript memory only (closure or React state). Never
localStorage— XSS-readable. - Refresh token:
httpOnly; Secure; SameSite=Strictcookie — inaccessible from JS. - On page reload:
prompt=nonesilent/authorizeround using the refresh token cookie.
Native mobile app:
- Access token: OS keychain (iOS Keychain, Android Keystore). Not process memory.
- Refresh token: OS keychain, never in
SharedPreferencesor unprotected storage.
Server-side app:
- Access token: server session (in-memory or Redis, not the client).
- Refresh token: server session. Client never sees either token.
Machine-to-machine (CI/CD, service):
- Use
client_credentialsgrant, not user-delegated tokens. - Never embed user access tokens in CI pipelines.
Production observability
A minimum OAuth observability dashboard:
| Metric | Alert condition |
|---|---|
refresh_replay_detected_total | Any non-zero value — probable compromise |
id_token_validation_failure_total by reason | Spike on sig/kid → JWKS rotation issue |
token_introspection_latency_p99 | Above 200ms → IdP overloaded |
jwks_cache_hit_ratio | Drop below 90% → cache TTL too short |
dpop_proof_failure_total by reason | Spike on iat_skew → clock drift in mobile clients |
token_request_total by outcome | Spike on invalid_grant → attack or misconfigured client |
refresh_replay_detected_total > 0 is the most actionable alert: it means at least one refresh token was used by two distinct clients — someone stole a token.
Four real-world OAuth failures
Facebook, September 2018. A bug in the “View As” feature exposed 50 million access tokens. Attackers could impersonate any of those users at any Facebook OAuth client (Spotify, Tinder, Instagram). Fix: invalidate 90 million tokens.
Slack, February 2017. A misconfigured OAuth redirect URI in a third-party Slack app let an attacker phish a workspace owner’s authorization code. The attacker escalated to full admin access. Fix: URI validation, required redirect URI exact-match enforcement at the IdP level.
GitHub Enterprise, 2021. A bug in PKCE verification allowed downgrade attacks against older clients. Fixed in a patch release.
Microsoft Azure AD, November 2024. A token-validation cache poisoning vulnerability let a crafted token bypass aud validation for ~30 minutes on cached negative responses. Fixed within hours. Root cause: the negative-response cache did not distinguish “aud not found” from “aud explicitly rejected.”
The pattern: every incident involved one skipped or buggy mandatory check. The industry response to all four was the same — more mandatory checks, shorter TTLs, wider observability.
A team wants to share one access token between two resource servers (API-A and API-B) for convenience. Why is this dangerous?
Why should access tokens never be stored in localStorage in a browser SPA?
Order the steps to diagnose a spike in id_token validation failures:
- 1 Check the failure_reason dimension on id_token_validation_failure_total
- 2 If reason is 'sig' or 'kid-not-found', suspect JWKS key rotation
- 3 Check jwks_cache_hit_ratio — a drop indicates stale cache from rotation
- 4 Confirm by checking if the IdP published a new signing key recently
- 5 Trigger immediate JWKS refresh across all instances
- 6 Reduce JWKS cache TTL to 5–10 minutes and add on-cache-miss refresh as backstop
- 01Explain the audience confusion attack and the correct aud check that prevents it.
- 02Why is refresh_replay_detected_total > 0 a compromise indicator rather than a harmless error?
- 03What observability gap does short access token TTL (5–15 min) address?
Audience validation is a hard-coded identity check, not a membership check — the token’s aud must contain this resource server’s specific identifier, never derived from the request. Token storage follows the client’s threat model: JS memory for SPAs, OS keychain for native, server session for server-side. Production observability must expose refresh_replay_detected_total (compromise signal), JWKS cache hit ratio (rotation readiness), and introspection latency (IdP health). The four major OAuth incidents — Facebook 2018, Slack 2017, GitHub Enterprise 2021, Azure AD 2024 — each followed from one skipped or buggy mandatory check. OAuth security is a complete-set problem: every check must pass, every time. Now when you see a team tempted to relax the aud check “just for integration testing,” you’ll know what to say: that is not a test shortcut — it is the exact misconfiguration behind $100M breach post-mortems.
Practice
Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.
appears again in228
- Why GraphQL gets N+1junior
- DataLoader mechanics: tick-boundary batchingmiddle
- Batch function contracts: ordering, shapes, errorsmiddle
- Federation and lookahead: batching beyond DataLoadermiddle
- Query complexity defences: depth, cost, persisted queriesmiddle
- Senior GraphQL API: scheduling contract, tenant isolation, observabilitysenior
- Why idempotency: making retries safejunior
- Server-side state machine: four states of an idempotency keymiddle
- Outbox and inbox: effectively-once across the dual-write boundarymiddle
- Concurrency and cache architecture for idempotency at scalesenior
- Observability, production failures, and global-scale designsenior
- The event loop: one thread, three queuesjunior
- Tasks, microtasks, and scheduler.yield()middle
- Microtask starvation, Long Tasks, and LoAFsenior
- Node.js event loop: phases, nextTick, and loop lagsenior
- React, Vue, and INP observability in productionsenior
- The render pipeline: six stages from bytes to pixelsjunior
- Stage costs and the renderer process modelmiddle
- Invalidation, dirty bits, and containmiddle
- Compositor layers: promotion, overlap, and GPU memorymiddle
- DevTools flame strip and the frame lifecyclemiddle
- Layout thrash: forced synchronous layoutsenior
- BeginMainFrame, compositor-driven animations, and GPU memorysenior
- Production observability: LoAF, INP, and the full attack surfacesenior
- What V8 is and why performance varies 100×junior
- V8''''s four-tier JIT pipeline and profile-guided tieringmiddle
- Hidden classes, transition trees, and memory layoutmiddle
- Inline caches, IC states, and deoptimizationmiddle
- Orinoco GC: parallel scavenger, concurrent marking, and write barriersmiddle
- TurboFan''''s speculative engine and the deopt-loop trapsenior
- V8 in production: isolates, pointer compression, and real failuressenior
- What workers are and why they existjunior
- Web worker mechanics: dedicated, shared, and OffscreenCanvasmiddle
- Structured clone and transferablesmiddle
- Service worker lifecycle and cache strategiesmiddle
- SharedArrayBuffer, Atomics, and cross-origin isolationsenior
- Service worker edge cases: version skew, durability, and navigation trapssenior
- Worker pools, Comlink, and production observabilitysenior
- What the reconciler does: render vs commitjunior
- The fiber object and the double-buffer treemiddle
- Render phase purity and commit phase sub-stepsmiddle
- Reconciliation: diffing heuristics and the key trapmiddle
- Priority lanes, time-slicing, and useTransitionmiddle
- Bailout, memoisation, and tearingsenior
- React Profiler, the Compiler, and production observabilitysenior
- Rendering strategies: SSG, SSR, ISR, streaming, and hydrationjunior
- SSG, SSR, ISR, streaming, and RSC — how each worksmiddle
- Hydration cost: selective, progressive, islands, resumabilitymiddle
- Hydration mismatch: causes, detection, and the determinism rulesenior
- RSC, per-route strategy, and production observabilitysenior
- Core Web Vitals: what LCP, INP, and CLS measurejunior
- CLS: why layout shifts happen and how to stop themmiddle
- Metric tradeoffs, RUM attribution, and the CI+field loopsenior
- The full picture: URL to LCP to INP as a relay racejunior
- Eight layers traced: from the service worker to the second navigationmiddle
- Five canonical breaks: where production reliably diessenior
- The three-track method: reading traces and building a monitored systemsenior
- What is a cache stampede and why it makes things worsejunior
- Lock and single-flight: bounding concurrent rebuildsmiddle
- XFetch: coordination-free probabilistic early expirationmiddle
- Stale-while-revalidate and CDN request coalescingmiddle
- Detecting stampedes and designing TTL for productionmiddle
- Metastable failure, fencing tokens, and production postmortemssenior
- What a relation is: tables, rows, keys, and constraintsjunior
- Constraints, keys, and Postgres data typesmiddle
- Normal forms, denormalization, and why schemas stickmiddle
- JSONB, arrays, and when a side table winsmiddle
- Heap storage, TOAST, and column alignmentsenior
- Schema integrity: deferral, versioning, and production failure modessenior
- Relational vs document, wide-column, graph, and key-valuesenior
- Index-only scans, the Visibility Map, and INCLUDEsenior
- Production failure modes and the index audit playbooksenior
- pg_statistic, ANALYZE, and production observabilitymiddle
- Production failure modes and plan stabilitysenior
- MVCC: why readers and writers never wait for each otherjunior
- Row versions and snapshots: the on-disk mechanicsmiddle
- HOT updates and isolation levels: what you gain and what you paymiddle
- Vacuum and bloat: keeping the storage tax boundedmiddle
- CLOG, XID wraparound, and MultiXact: deep visibility internalssenior
- SSI internals and production autovacuum tuningsenior
- Real-world MVCC failures, deployment patterns, and distributed snapshotssenior
- Connection pools: amortising the cost of a Postgres backendjunior
- PgBouncer session, transaction, and statement modesmiddle
- Pool sizing: the (cores × 2) + spindles formula and the two-layer stackmiddle
- Pool exhaustion and idle-in-transaction: the 3 AM failure modemiddle
- Migrating to transaction mode: rollout playbook and PgBouncer 1.21 prepared statementsmiddle
- The Postgres process model and why raising max_connections degrades throughputsenior
- Pooler landscape 2026, serverless connection storms, and the full failure-mode taxonomysenior
- What a schema migration is and why it replaces ad-hoc DDLjunior
- ADD COLUMN: instant in PG 11+ vs rewrite in older Postgresjunior
- The lock-queue failure mode: why instant DDL can freeze the databasemiddle
- Safe DDL patterns: NOT VALID, CONCURRENTLY, and unsafe-op fixesmiddle
- Expand-contract: zero-downtime for breaking schema changesmiddle
- Advisory locks, migration tools, and deploy coordinationsenior
- Migration failure taxonomy and production disciplinesenior
- Why sharding exists: the single-Postgres ceilingjunior
- Shard-key selection: hash, range, list, and directory strategiesmiddle
- Partitioning vs sharding: same word, two different thingsmiddle
- Co-location and Citus: the invariant that makes sharding usablemiddle
- The hot-shard failure mode: detection, isolation, and durable policymiddle
- Schema-based sharding and multi-tenancy alternativessenior
- Online resharding, 2PC, and the operational cost of shardingsenior
- The seven acts: from CREATE TABLE to Citusjunior
- Acts 1–3 in depth: schema, indexes, and planner statisticsmiddle
- Acts 4–6 in depth: MVCC bloat, connection pooling, and safe migrationsmiddle
- Act 7 in depth: sharding, co-location, and the seven-tier tradeoff cascademiddle
- Observability, anti-patterns, and production triagesenior
- Raft roles, terms, and why majority quorums prevent split brainjunior
- How Raft replicates a log entry and decides it is safe to commitmiddle
- Raft leader election: timeouts, voting rules, and the four safety propertiesmiddle
- Raft in the real world: partitions, slow disks, and client routingmiddle
- Raft extensions: pre-vote, learners, snapshots, and linearizable readssenior
- Raft in production: membership changes, Multi-Raft, and observabilitysenior
- Where data fetching happens — and why it decides LCPjunior
- Fetch waterfalls — diagnosis and the Promise.all curemiddle
- React Server Components and Suspense streamingmiddle
- Client-side cache: TanStack Query, SWR, and stale-while-revalidatemiddle
- LCP, prefetch, and race conditions in interactive fetchingmiddle
- Senior internals: RSC payload, caching layers, and production failure modessenior
- What a JavaScript engine iszero
- Ignition and the bytecodemiddle
- Inside the interpreter loopmiddle
- How V8 represents a valuemiddle
- What a Map really is: opening the hidden classmiddle
- Transition trees, deprecation, and migrationmiddle
- Fast properties, slow properties, and slack trackingmiddle
- Inline caches, deeply: feedback slots and handlersmiddle
- Monomorphic, polymorphic, megamorphic — and the stub cachemiddle
- Four tiers and how warm-up worksmiddle
- Type feedback: the fuel for the optimisersenior
- TurboFan: the sea-of-nodes optimisersenior
- Speculation and guardssenior
- Deoptimization: falling off the cliffsenior
- On-Stack Replacement: swapping the frame mid-loopsenior
- Scopes, contexts, and the scope chainmiddle
- What a closure actually retainsmiddle
- When V8 allocates a Context (and what it costs)middle
- Closures, feedback vectors, and call-site polymorphismmiddle
- Why a GC, and reachabilitymiddle
- Leaks in a garbage-collected languagesenior
- The engine vs the loopmiddle
- Microtasks vs macrotasks: orderingmiddle
- Inside a Promisemiddle
- async/await, desugaredmiddle
- Capstone: optimize a hot pathsenior
- The IP envelopejunior
- Reading the IP headermiddle
- The three-way handshakejunior
- Sequence numbers and connection statemiddle
- DNS: what it does and why it existsjunior
- The resolver walk: referrals, record types, and gluemiddle
- TTL, caching, and DNS propagationmiddle
- What TLS does and why it existsjunior
- The 1-RTT handshake: key shares and ECDHEmiddle
- Session resumption and 0-RTTmiddle
- Key schedule, SNI, ALPN, and extensionssenior
- 0-RTT defenses, ECH, hybrid PQ, and production TLSsenior
- WebSocket: the HTTP upgrade handshakejunior
- WebSocket frame format: opcodes, masking, fragmentationmiddle
- WebSocket backpressure: when clients can''''t keep upmiddle
- Reconnection: jittered backoff, thundering herd, message resumptionsenior
- WebSocket at scale: HTTP/2 multiplexing, permessage-deflate, C10Msenior
- WebSocket in production: proxies, security, and distributed architecturesenior
- What reverse proxies dojunior
- Health checks, connection draining, and slow startmiddle
- Session affinity, consistent hashing, and the right fixmiddle
- Retry storms, circuit breakers, and load sheddingsenior
- Resilient LB architecture: anycast, zone-aware routing, and observabilitysenior
- Why QUIC and not TCP+TLSjunior
- Connection IDs and network migrationmiddle
- 0-RTT resumption and packet encryptionsenior
- DDoS: what it is and why it worksjunior
- Amplification attacks and state exhaustionmiddle
- Rate limiting: algorithms and architecturemiddle
- WAFs, firewalls, mTLS, and HSTSmiddle
- DNS cache poisoning and BGP hijackingsenior
- Defense-in-depth architecture and attack economicssenior
- The twelve layers: one URL, seven actorsjunior
- DNS, TCP, TLS in sequence: where the milliseconds gomiddle
- Proxy intercepts and security gates: rate limiters, WAF, mTLSmiddle
- Alternate paths: QUIC 0-RTT, WebSocket upgrade, connection migrationmiddle
- Observability: distributed traces, USE/RED, and samplingsenior
- Resilience: cascading retries, circuit breakers, and error budgetssenior
- What the three signals are: logs, metrics, and tracesjunior
- Why structured logs exist: the diary vs the spreadsheetjunior
- The production log schema: fields every line must carrymiddle
- PII redaction and log injectionsenior
- OTel Logs Data Model and audit logs as a subsystemsenior
- What is OpenTelemetry: API, SDK, Collector, OTLPjunior
- OTel signals, Semantic Conventions, and the OTLP wire formatmiddle
- The OTel Collector: receivers, processors, exporters, and deployment patternsmiddle
- Vendor neutrality, eBPF instrumentation, the Operator, and browser/serverless OTelsenior
- Operating the OTel Collector: reliability, version skew, failure modes, and governancesenior
- SLI, SLO, and the error budget: reliability by the numbersjunior
- Error budget policy, latency SLOs, and composite journeysmiddle
- Production SLO failures, self-observability, security, and the big picturesenior
- What is trace propagation and why broken propagation is worse than nonejunior
- traceparent and tracestate: the W3C header format in fullmiddle
- Baggage and async boundaries: carrying context across queues and callbacksmiddle
- Async context per language, service mesh, B3 migration, and securitysenior
- Production propagation failures, span links, and platform designsenior
- The debugging funnel: SLO → RED → trace → profilejunior
- OTel architecture: one SDK, four signals, one wire formatmiddle
- The incident loop: from pager to postmortem to preventionmiddle
- Scale, security, and the ROI of observable systemssenior
- Cache lines, struct layout, and false sharingmiddle
- SIMD, SoA vs AoS, and memory bandwidthmiddle
- Cache-oblivious algorithms, PGO, and production failuressenior
- GC in production: observability, security, edge cases, and fleet governancesenior
- Batching: amortize fixed cost per operationjunior
- The batching window: size and wait timemiddle
- Batching in Kafka and Postgresmiddle
- io_uring and observability of batchingmiddle
- From Nagle to io_uring: evolution of batchingmiddle
- Backpressure, failure isolation, and batch security in productionsenior
- CI enforcement and RUM: making budgets stickmiddle
- V8 JIT pipeline, HTTP priorities, and bundle securitysenior
- The performance loop: discipline, not a projectjunior
- Classify and fix: matching bottleneck families to remediesmiddle
- Observability stack and CI gates: catching regressions before they shipmiddle
- Incident to enforcement: SLO burn to verified fix in 35 minutesmiddle
- Culture, economics, and org-scale performancesenior
- At-most-once, at-least-once, exactly-once: the three delivery contractsjunior
- The three failure legs — where duplicates and losses actually happenmiddle
- Consumer-side dedup: the cheapest path to exactly-once processingmiddle
- Kafka exactly-once semantics: idempotent producer and transactionsmiddle
- SQS visibility timeout, DLQ, and the outbox patternmiddle
- Exactly-once in production: impossibility proof, hybrid patterns, and real incidentssenior
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.