GC algorithms: generational, concurrent, and per-runtime
The generational hypothesis (most objects die young) shapes every major production GC. Knowing which algorithm your runtime ships — and why it makes different tradeoffs than others — is the prerequisite for diagnosing any pause.
A Go service and a JVM service are both running at 200 MB/s allocation rate. The Go service has flat 0.5 ms pauses; the JVM service has 50 ms pauses every 10 seconds. Same allocation rate, 100x pause difference — because the collectors work entirely differently. By the end of this lesson you will know exactly why that gap exists and which collector design fits your runtime.
The generational hypothesis
The single most important empirical observation in GC research: most objects die young. A typical request handler allocates many temporary objects — parsed body, intermediate strings, response builders — serves the response, and drops them all. The objects that survive long enough to be useful are few: caches, configuration, connection pools.
Generational collectors exploit this by splitting the heap:
- Young generation — small (a few hundred MB to a few GB), collected frequently. All allocations land here.
- Old generation — large, collected rarely. Objects that survive several young-gen cycles get “promoted” (tenured) here.
A young-gen collection walks only the small young heap. If 95% of objects die in young, one cheap pass reclaims 95% of allocated bytes. Old-gen collections are expensive but rare because most objects never reach old.
| Generation | Size | Collection frequency | Contents |
|---|---|---|---|
| Young (Eden + Survivor) | ~256 MB – 4 GB | Every few seconds | New allocations; mostly short-lived |
| Old (Tenured) | Remaining heap | Every few minutes | Promoted survivors; mostly long-lived |
Concurrent marking
To keep pauses below 10 ms on multi-GB heaps, marking must happen while the application runs. The GC uses a background thread that walks the object graph alongside mutator (application) threads.
The challenge: the application may modify references while marking is in progress, so the GC needs a write barrier — a small piece of code that runs on every reference write to keep the collector informed. Two main strategies:
- Snapshot-at-the-beginning (SATB) — marks the old reference about to be overwritten so the collector treats it as live for this cycle. Used by G1, Shenandoah, ZGC.
- Incremental-update — marks the new reference being written so it is not missed. Used by CMS, classic V8 mark-compact.
Write barrier overhead: ~2–10% CPU — the price of concurrent marking.
Per-runtime tour
When you tune or diagnose GC in production, the collector your runtime ships is the constraint you cannot change — so knowing its design is not optional.
Go — concurrent tri-color non-generational mark-sweep (no compaction; allocator is a tcmalloc variant). Triggered by allocation rate; the pacer keeps STW pauses under 1 ms. GOGC=100 means GC runs when the heap doubles since the last cycle; GOMEMLIMIT (Go 1.19+) adds a soft memory cap. Go is the counterexample to generational: the team bet that simpler runtime + escape analysis reducing heap pressure made the generational complexity not worth it. Empirically validated — sub-ms pauses on most workloads.
JVM (modern) — G1 is the default for most servers: region-based, generational, concurrent marking, typical pauses 10–50 ms. ZGC (JEP 333 experimental JDK 11, production JDK 15; JEP 377) targets sub-ms pauses on heaps up to 16 TB using colored pointers + load barriers. Shenandoah (Red Hat) uses Brooks pointers for similar goals. Tunable via -XX:MaxGCPauseMillis and heap-size flags.
V8 (Node.js) — generational Scavenger for young (semi-space copying), Mark-Compact for old. The Orinoco project (2017+) added concurrent marking + parallel compaction, targeting ≤10 ms pauses. Heap capped per-isolate (~1.5 GB default in Node, configurable via --max-old-space-size).
.NET — workstation/server GC, generational (gen 0/1/2 + LOH), background concurrent marking. Tunable via GCSettings.LatencyMode.
CPython — reference counting (drops object when refcount=0, no major pause) + cycle collector for reference cycles. The GIL serialises mutation but reference counts cost ~10–20% throughput. No major STW pauses; cycle collection is incremental.
▸Why this works
The generational hypothesis still informs production even in non-generational runtimes like Go. Code that allocates many short-lived objects in tight loops — zero-capacity slices that grow, fmt.Sprintf on every request, JSON encoding without pooling — creates more GC pressure than long-lived state regardless of the collector’s algorithm. The lever is always the same: reduce short-lived allocations.
A JVM service has 50 ms p99 from G1 pauses. Doubling the heap (-Xmx 8g → 16g) might help — why, and what is the risk?
A V8 service running in Node.js grows to 1.4 GB then crashes. Most likely cause?
A Python service is single-threaded due to the GIL, but still has GC overhead. What is the dominant cost?
Order the priority of GC-pressure fix levers, from highest leverage to lowest:
- 1 Eliminate the allocation (in-place mutation, struct-of-arrays, primitives)
- 2 Pool / reuse the allocation (sync.Pool, ObjectPool, bytes.Buffer reset)
- 3 Let escape analysis stack-allocate (smaller object, scope-local)
- 4 Shrink the allocation (smaller struct, smaller buffer, pre-sized container)
- 5 Move the allocation off the hot path (cache the result, compute once)
- 6 Tune the collector (GOGC, MaxGCPauseMillis, max-old-space-size)
- 7 Switch the collector algorithm (ParallelGC → G1 → ZGC)
- 01Walk through the generational hypothesis and why it shapes most production GCs. Include one counterexample.
- 02What is a write barrier in concurrent GC and why is it needed?
The generational hypothesis — most objects die young — is the empirical foundation of most production GC algorithms. Generational collectors split the heap into a small, frequently-collected young generation and a large, rarely-collected old generation; most garbage is reclaimed cheaply in the young gen. Concurrent marking runs alongside the application by adding a write barrier (~2–10% CPU) to intercept reference writes. Go is the main counterexample: non-generational, concurrent tri-color, betting that escape analysis reduces heap pressure enough. JVM G1 and ZGC, V8 Orinoco, and .NET server GC are all generational + concurrent. CPython is reference-counting with a supplementary cycle collector — no major STW pauses but per-operation refcount overhead. Now when you see a GC pause profile you have not encountered before, your first question should be: which generation did this collection target, and was concurrent marking able to run?
Practice
Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.
appears again in162
- The journey of a request: seven stops from socket to responsejunior
- Accept and parse: from kernel queue to a typed requestmiddle
- Routing and middleware: choosing what runs, and in what ordermiddle
- Handler and response: from business logic to bytes on the wiremiddle
- Streaming and backpressure: when the client reads slower than you writesenior
- Timeouts and tail latency: budgets, deadlines, and the fan-out trapsenior
- Middleware and DI: the two patterns that shape every backendjunior
- Writing middleware: signatures, next(), and the three framework modelsmiddle
- Inversion of control: how dependencies reach a classmiddle
- DI scopes and lifecycles: singleton, request, transientmiddle
- DI as a testing seam: fakes, mocks, and the boundary that matterssenior
- DI containers in production: resolution graphs, circular deps, and when not tosenior
- Blocking vs non-blocking I/O: two ways to waitjunior
- The event loop: one thread, ordered phasesmiddle
- What blocks the loop: CPU work and sync callsmiddle
- Offloading CPU work: worker threads and the libuv poolmiddle
- Backpressure and bounded concurrencysenior
- Throughput under load: tail latency and saturationsenior
- Why pool: the cost of creating a connectionjunior
- Pool sizing: why bigger is not fastermiddle
- Acquisition and timeouts: the wait queue is the real latency dialmiddle
- Retry strategies: backoff, jitter, and thundering herdmiddle
- Observability, production failures, and global-scale designsenior
- Tasks, microtasks, and scheduler.yield()middle
- Timer accuracy, throttling, and idle workmiddle
- Node.js event loop: phases, nextTick, and loop lagsenior
- Rendering strategies: SSG, SSR, ISR, streaming, and hydrationjunior
- SSG, SSR, ISR, streaming, and RSC — how each worksmiddle
- Hydration cost: selective, progressive, islands, resumabilitymiddle
- Core Web Vitals: what LCP, INP, and CLS measurejunior
- LCP: four phases, one dominant costmiddle
- INP: input delay, processing, presentationmiddle
- Lab vs field: why the two disagree and how to use eachmiddle
- Metric tradeoffs, RUM attribution, and the CI+field loopsenior
- The full picture: URL to LCP to INP as a relay racejunior
- Eight layers traced: from the service worker to the second navigationmiddle
- Five canonical breaks: where production reliably diessenior
- The three-track method: reading traces and building a monitored systemsenior
- What an index is and how it speeds up queriesjunior
- The leading-column rule and composite index designmiddle
- Partial, expression, and covering indexesmiddle
- Index types: GIN, GiST, BRIN, Hash, Bloom, and HOT updatesmiddle
- Index-only scans, the Visibility Map, and INCLUDEsenior
- Production failure modes and the index audit playbooksenior
- Index design exercise: full-text search strategysenior
- EXPLAIN and execution plans: what the planner decides and whyjunior
- Scan types: Seq, Index, Bitmap, Index-Onlymiddle
- Join algorithms and the row-estimate cascademiddle
- pg_statistic, ANALYZE, and production observabilitymiddle
- Extended statistics: fixing correlated-column estimate failuressenior
- Plan cache, cost-constant tuning, and planner internalssenior
- Production failure modes and plan stabilitysenior
- Connection pools: amortising the cost of a Postgres backendjunior
- PgBouncer session, transaction, and statement modesmiddle
- Pool sizing: the (cores × 2) + spindles formula and the two-layer stackmiddle
- Pool exhaustion and idle-in-transaction: the 3 AM failure modemiddle
- Migrating to transaction mode: rollout playbook and PgBouncer 1.21 prepared statementsmiddle
- The Postgres process model and why raising max_connections degrades throughputsenior
- Pooler landscape 2026, serverless connection storms, and the full failure-mode taxonomysenior
- ADD COLUMN: instant in PG 11+ vs rewrite in older Postgresjunior
- The lock-queue failure mode: why instant DDL can freeze the databasemiddle
- Safe DDL patterns: NOT VALID, CONCURRENTLY, and unsafe-op fixesmiddle
- Migration failure taxonomy and production disciplinesenior
- Shard-key selection: hash, range, list, and directory strategiesmiddle
- Co-location and Citus: the invariant that makes sharding usablemiddle
- The hot-shard failure mode: detection, isolation, and durable policymiddle
- Online resharding, 2PC, and the operational cost of shardingsenior
- The seven acts: from CREATE TABLE to Citusjunior
- Acts 1–3 in depth: schema, indexes, and planner statisticsmiddle
- Acts 4–6 in depth: MVCC bloat, connection pooling, and safe migrationsmiddle
- Act 7 in depth: sharding, co-location, and the seven-tier tradeoff cascademiddle
- Observability, anti-patterns, and production triagesenior
- Generational GC and the Scavengermiddle
- Major GC: mark, sweep, compactsenior
- Write barriers: the price of incremental and generational GCsenior
- Bits on the wirejunior
- Latency mathmiddle
- Bufferbloat and congestionsenior
- The physical frontiersenior
- Sequence numbers and connection statemiddle
- Flow control and congestion controlmiddle
- BBR, production observability, and beyond TCPsenior
- CDN: putting content next doorjunior
- Anycast and GeoDNS: routing to the nearest edgemiddle
- Tiered cache and Cache-Controlmiddle
- Vary header and cache keysmiddle
- Stale-while-revalidate and cache stampedesenior
- Edge workers and edge-side compositionsenior
- CDN operations and observabilitysenior
- WebSocket: the HTTP upgrade handshakejunior
- WebSocket vs SSE vs long-polling: choosing the right transportmiddle
- WebSocket backpressure: when clients can''''t keep upmiddle
- Reconnection: jittered backoff, thundering herd, message resumptionsenior
- WebSocket at scale: HTTP/2 multiplexing, permessage-deflate, C10Msenior
- WebSocket in production: proxies, security, and distributed architecturesenior
- What reverse proxies dojunior
- Balancing algorithms: round-robin to power-of-two-choicesmiddle
- L4 vs L7 load balancing and client-IP preservationmiddle
- Health checks, connection draining, and slow startmiddle
- Retry storms, circuit breakers, and load sheddingsenior
- Resilient LB architecture: anycast, zone-aware routing, and observabilitysenior
- Why QUIC and not TCP+TLSjunior
- QUIC streams and head-of-line blockingjunior
- Integrated handshake and 1-RTTmiddle
- Connection IDs and network migrationmiddle
- Loss detection and congestion controlmiddle
- 0-RTT resumption and packet encryptionsenior
- Deployment tradeoffs and CPU costsenior
- DDoS: what it is and why it worksjunior
- Amplification attacks and state exhaustionmiddle
- Rate limiting: algorithms and architecturemiddle
- WAFs, firewalls, mTLS, and HSTSmiddle
- DNS cache poisoning and BGP hijackingsenior
- Defense-in-depth architecture and attack economicssenior
- The twelve layers: one URL, seven actorsjunior
- DNS, TCP, TLS in sequence: where the milliseconds gomiddle
- Critical render path and Core Web Vitalsmiddle
- Proxy intercepts and security gates: rate limiters, WAF, mTLSmiddle
- Alternate paths: QUIC 0-RTT, WebSocket upgrade, connection migrationmiddle
- Observability: distributed traces, USE/RED, and samplingsenior
- Resilience: cascading retries, circuit breakers, and error budgetssenior
- What the three signals are: logs, metrics, and tracesjunior
- Metrics and cardinality: the cost model of a time-series databasemiddle
- Logs and volume: the cost model of structured loggingmiddle
- Traces and sampling: the cost model of distributed tracingmiddle
- Join keys and exemplars: making the three signals composemiddle
- Observability 2.0: wide events and the cost shiftsenior
- Failure modes and engineering practice: cardinality budgets, PII, and samplingsenior
- Why structured logs exist: the diary vs the spreadsheetjunior
- The production log schema: fields every line must carrymiddle
- Log levels and alert routingmiddle
- Sampling strategies and log costmiddle
- PII redaction and log injectionsenior
- Trace context propagation in logssenior
- OTel Logs Data Model and audit logs as a subsystemsenior
- OTel signals, Semantic Conventions, and the OTLP wire formatmiddle
- Auto-instrumentation and manual spans: the 80/20 of OTelmiddle
- The OTel Collector: receivers, processors, exporters, and deployment patternsmiddle
- Sampling strategies: head, tail, and parent-basedmiddle
- Vendor neutrality, eBPF instrumentation, the Operator, and browser/serverless OTelsenior
- Operating the OTel Collector: reliability, version skew, failure modes, and governancesenior
- RED and USE: two checklists, one triage disciplinejunior
- Instrumenting RED in Prometheus: counters, histograms, and cardinality disciplinemiddle
- USE on Linux: CPU, memory, disk, network, and PSImiddle
- Golden signals, dashboard layout, and service mesh auto-REDmiddle
- Cardinality as a cost driver: labels, PII, exemplars, and samplingmiddle
- Native histograms, SLO tie-in, and production failure patternsmiddle
- Choosing SLIs and SLO targets: ratios, not feelingsmiddle
- Multi-window multi-burn-rate alerting: why AND beats ORmiddle
- Error budget policy, latency SLOs, and composite journeysmiddle
- Iceberg SLIs, composite SLO math, and SLA vs SLOsenior
- Flame graphs: reading the picture that shows where time goesjunior
- Sampling vs instrumentation profiling: why 99 Hz wins in productionmiddle
- Profile types: CPU, memory, off-CPU, mutex — which one to reach formiddle
- Continuous profiling: always-on flame graphs with eBPF and trace-id correlationmiddle
- How flame graphs are built from samples, and the production workflows that use themmiddle
- Linux perf, eBPF internals, PGO, and the limits of samplingsenior
- Profiling in production: security, war stories, OTel profiles, and the infrastructure designsenior
- The debugging funnel: SLO → RED → trace → profilejunior
- OTel architecture: one SDK, four signals, one wire formatmiddle
- Cost discipline: keeping observability under 5% of infra spendmiddle
- Scale, security, and the ROI of observable systemssenior
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.