Databases
Scan types: Seq, Index, Bitmap, Index-Only
EXPLAIN shows Seq Scan on your 10M-row table even though an index exists on the filter column. The planner is not broken — it calculated that a sequential scan is cheaper for this selectivity. Understanding when and why the planner picks each scan type is what separates guessing from diagnosing.
Reading the cost line
Every plan node prints:
(cost=startup..total rows=N width=B)startup— cost paid before the first row can be emitted. Zero for Seq Scan; non-zero for Sort (must read everything first).total— cost to run the node to completion.rows— estimated rows emitted by this node.width— average row width in bytes.
Cost is in arbitrary units where 1.0 ≈ one sequential page fetch. Only ratios matter; the absolute numbers are meaningless without context. With ANALYZE, every node also gets (actual time=startup..total rows=N loops=L) — real wall time in milliseconds.
Interpretation rule: if the node is inside a Nested Loop’s inner side, divide actual time by loops to get per-execution cost.
The four scan types
| Scan type | I/O pattern | Wins when |
|---|---|---|
| Seq Scan | Full heap, sequential | Predicate matches many rows, or no usable index |
| Index Scan | Index walk + heap fetch per row (random I/O) | Few rows match (<5-15% of table) and rows are not clustered |
| Bitmap Heap Scan | Index walk → bitmap of TIDs → sorted heap read | Many scattered rows (hundreds to millions); turns random I/O into sequential |
| Index Only Scan | Index leaves only — no heap fetch | All needed columns are in the index AND Visibility Map says pages are all-visible |
Seq Scan
Cost formula: relpages × seq_page_cost + reltuples × cpu_tuple_cost. Default seq_page_cost = 1.0, cpu_tuple_cost = 0.01. For a 5,000-page table: 5000 × 1.0 + 1M × 0.01 = 15,000 units. Optimal when the predicate is unselective (most rows match) — reading the whole heap sequentially is faster than hopping randomly to each matching row.
Index Scan
Cost grows with random_page_cost × matching_rows. Default random_page_cost = 4.0 (HDD-era). For 100 matching rows: 100 × 4.0 = 400 — much cheaper than the 15,000 Seq Scan. For 100,000 matching rows: 100,000 × 4.0 = 400,000 — Seq Scan wins. The crossover is roughly when the predicate matches 5–15% of the table.
SSD tuning note: on NVMe SSDs, random reads are only 1.5–2× slower than sequential — not 4×. Set random_page_cost = 1.1 in postgresql.conf for SSD systems. The planner will then prefer Index Scan for a much larger fraction of queries.
Bitmap Heap Scan
A two-step process: first a Bitmap Index Scan builds an in-memory bitmap of all matching TIDs from the index; then Bitmap Heap Scan sorts the TIDs by physical page address and reads the heap in order. This converts scattered random reads into a sequential heap traversal. It wins when many rows match but they are scattered across the heap — the middle ground between Index Scan and Seq Scan.
The bitmap can be OR-ed across multiple indexes (BitmapOr node) — useful when an OR predicate has an index per column.
Index Only Scan
Reads only the index leaf pages — no heap fetch — when two conditions are both true: (1) all columns needed by the query are stored in the index (either as key columns or via INCLUDE), and (2) the Visibility Map confirms the page is “all-visible” (every tuple visible to all transactions). When the VM bit is clear, Postgres must fall back to a heap fetch for that page. In EXPLAIN ANALYZE output, Heap Fetches: N reveals how often this happened — high N means the VM is stale and VACUUM has not run recently.
BUFFERS: the I/O picture
EXPLAIN (ANALYZE, BUFFERS) adds buffer accounting per node:
Seq Scan on orders (actual time=0.1..450 rows=1000000 loops=1)
Buffers: shared hit=4800 read=200shared hit— page found in Postgres’sshared_bufferscache (no disk I/O)read— page fetched from OS or diskdirtied— page modified by this nodewritten— page flushed to disk
High read with low hit means the working set is larger than shared_buffers, or the query is touching cold data. This diagnoses “is this slow because of CPU or I/O”: high reads = I/O-bound; low reads but high actual time = CPU-bound (often a poor join or heavy sort).
- seq_page_cost (default)
- 1.0
- random_page_cost (default, HDD-tuned)
- 4.0
- random_page_cost (SSD-tuned)
- 1.1
- cpu_tuple_cost
- 0.01
- cpu_index_tuple_cost
- 0.005
- Index Scan vs Seq Scan crossover (typical)
- ~5-15% of table rows
- Index Only Scan speedup vs Index Scan
- ~10-100x on hot reads
A 1M-row table has an index on `status`. A query `WHERE status = 'active'` returns 400,000 rows (40% of the table). What scan type will the planner most likely choose?
EXPLAIN ANALYZE shows `Index Only Scan ... Heap Fetches: 12000`. What does this mean?
On an NVMe SSD server you leave `random_page_cost` at the default 4.0. What production effect does this have?
- 01Explain why the planner picks Seq Scan on a 1M-row table when an index exists on the filter column.
- 02When does Bitmap Heap Scan win over both Index Scan and Seq Scan?
- 03What is the BUFFERS output in EXPLAIN ANALYZE and how do you use it to diagnose I/O vs CPU bottlenecks?
Postgres picks among four scan types based on cost: Seq Scan reads every heap page sequentially and wins when most rows match; Index Scan walks the B-tree and fetches heap rows individually, winning when the predicate is selective (roughly <10% of rows); Bitmap Heap Scan builds a sorted TID bitmap and reads the heap in order, winning in the middle range of selectivity; Index Only Scan skips the heap entirely when all needed columns are in the index and the Visibility Map is current. The planner uses cost constants — seq_page_cost, random_page_cost, cpu_tuple_cost — to compare options. On NVMe SSD, set random_page_cost = 1.1 so the planner correctly prefers Index Scan for selective predicates. Use EXPLAIN (ANALYZE, BUFFERS) to see whether a query is I/O-bound (high page reads) or CPU-bound (high actual time with few reads).
Practice
Do these to turn recognition into skill.
- Extended statistics: fixing correlated-column estimate failuressenior
- Plan cache, cost-constant tuning, and planner internalssenior
- Production failure modes and plan stabilitysenior
- Execution plans: diagnose and stabilise a slow querysenior
- Execution plans: multiple-choice reviewsenior
- Execution plans: free-recall reviewsenior
appears again in174
- The journey of a request: seven stops from socket to responsejunior
- Accept and parse: from kernel queue to a typed requestmiddle
- Routing and middleware: choosing what runs, and in what ordermiddle
- Handler and response: from business logic to bytes on the wiremiddle
- Streaming and backpressure: when the client reads slower than you writesenior
- Timeouts and tail latency: budgets, deadlines, and the fan-out trapsenior
- Middleware and DI: the two patterns that shape every backendjunior
- Writing middleware: signatures, next(), and the three framework modelsmiddle
- Inversion of control: how dependencies reach a classmiddle
- DI scopes and lifecycles: singleton, request, transientmiddle
- DI as a testing seam: fakes, mocks, and the boundary that matterssenior
- DI containers in production: resolution graphs, circular deps, and when not tosenior
- Blocking vs non-blocking I/O: two ways to waitjunior
- The event loop: one thread, ordered phasesmiddle
- What blocks the loop: CPU work and sync callsmiddle
- Offloading CPU work: worker threads and the libuv poolmiddle
- Backpressure and bounded concurrencysenior
- Throughput under load: tail latency and saturationsenior
- Why pool: the cost of creating a connectionjunior
- Pool sizing: why bigger is not fastermiddle
- Acquisition and timeouts: the wait queue is the real latency dialmiddle
- Retry strategies: backoff, jitter, and thundering herdmiddle
- Observability, production failures, and global-scale designsenior
- Tasks, microtasks, and scheduler.yield()middle
- Timer accuracy, throttling, and idle workmiddle
- Node.js event loop: phases, nextTick, and loop lagsenior
- Rendering strategies: SSG, SSR, ISR, streaming, and hydrationjunior
- SSG, SSR, ISR, streaming, and RSC — how each worksmiddle
- Hydration cost: selective, progressive, islands, resumabilitymiddle
- Core Web Vitals: what LCP, INP, and CLS measurejunior
- LCP: four phases, one dominant costmiddle
- INP: input delay, processing, presentationmiddle
- Lab vs field: why the two disagree and how to use eachmiddle
- Metric tradeoffs, RUM attribution, and the CI+field loopsenior
- The full picture: URL to LCP to INP as a relay racejunior
- Eight layers traced: from the service worker to the second navigationmiddle
- Five canonical breaks: where production reliably diessenior
- The three-track method: reading traces and building a monitored systemsenior
- Bits on the wirejunior
- Latency mathmiddle
- Bufferbloat and congestionsenior
- The physical frontiersenior
- Sequence numbers and connection statemiddle
- Flow control and congestion controlmiddle
- BBR, production observability, and beyond TCPsenior
- CDN: putting content next doorjunior
- Anycast and GeoDNS: routing to the nearest edgemiddle
- Tiered cache and Cache-Controlmiddle
- Vary header and cache keysmiddle
- Stale-while-revalidate and cache stampedesenior
- Edge workers and edge-side compositionsenior
- CDN operations and observabilitysenior
- WebSocket: the HTTP upgrade handshakejunior
- WebSocket vs SSE vs long-polling: choosing the right transportmiddle
- WebSocket backpressure: when clients can''''t keep upmiddle
- Reconnection: jittered backoff, thundering herd, message resumptionsenior
- WebSocket at scale: HTTP/2 multiplexing, permessage-deflate, C10Msenior
- WebSocket in production: proxies, security, and distributed architecturesenior
- What reverse proxies dojunior
- Balancing algorithms: round-robin to power-of-two-choicesmiddle
- L4 vs L7 load balancing and client-IP preservationmiddle
- Health checks, connection draining, and slow startmiddle
- Retry storms, circuit breakers, and load sheddingsenior
- Resilient LB architecture: anycast, zone-aware routing, and observabilitysenior
- Why QUIC and not TCP+TLSjunior
- QUIC streams and head-of-line blockingjunior
- Integrated handshake and 1-RTTmiddle
- Connection IDs and network migrationmiddle
- Loss detection and congestion controlmiddle
- 0-RTT resumption and packet encryptionsenior
- Deployment tradeoffs and CPU costsenior
- DDoS: what it is and why it worksjunior
- Amplification attacks and state exhaustionmiddle
- Rate limiting: algorithms and architecturemiddle
- WAFs, firewalls, mTLS, and HSTSmiddle
- DNS cache poisoning and BGP hijackingsenior
- Defense-in-depth architecture and attack economicssenior
- The twelve layers: one URL, seven actorsjunior
- DNS, TCP, TLS in sequence: where the milliseconds gomiddle
- Critical render path and Core Web Vitalsmiddle
- Proxy intercepts and security gates: rate limiters, WAF, mTLSmiddle
- Alternate paths: QUIC 0-RTT, WebSocket upgrade, connection migrationmiddle
- Observability: distributed traces, USE/RED, and samplingsenior
- Resilience: cascading retries, circuit breakers, and error budgetssenior
- What the three signals are: logs, metrics, and tracesjunior
- Metrics and cardinality: the cost model of a time-series databasemiddle
- Logs and volume: the cost model of structured loggingmiddle
- Traces and sampling: the cost model of distributed tracingmiddle
- Join keys and exemplars: making the three signals composemiddle
- Observability 2.0: wide events and the cost shiftsenior
- Failure modes and engineering practice: cardinality budgets, PII, and samplingsenior
- Why structured logs exist: the diary vs the spreadsheetjunior
- The production log schema: fields every line must carrymiddle
- Log levels and alert routingmiddle
- Sampling strategies and log costmiddle
- PII redaction and log injectionsenior
- Trace context propagation in logssenior
- OTel Logs Data Model and audit logs as a subsystemsenior
- OTel signals, Semantic Conventions, and the OTLP wire formatmiddle
- Auto-instrumentation and manual spans: the 80/20 of OTelmiddle
- The OTel Collector: receivers, processors, exporters, and deployment patternsmiddle
- Sampling strategies: head, tail, and parent-basedmiddle
- Vendor neutrality, eBPF instrumentation, the Operator, and browser/serverless OTelsenior
- Operating the OTel Collector: reliability, version skew, failure modes, and governancesenior
- RED and USE: two checklists, one triage disciplinejunior
- Instrumenting RED in Prometheus: counters, histograms, and cardinality disciplinemiddle
- USE on Linux: CPU, memory, disk, network, and PSImiddle
- Golden signals, dashboard layout, and service mesh auto-REDmiddle
- Cardinality as a cost driver: labels, PII, exemplars, and samplingmiddle
- Native histograms, SLO tie-in, and production failure patternsmiddle
- Choosing SLIs and SLO targets: ratios, not feelingsmiddle
- Multi-window multi-burn-rate alerting: why AND beats ORmiddle
- Error budget policy, latency SLOs, and composite journeysmiddle
- Iceberg SLIs, composite SLO math, and SLA vs SLOsenior
- Flame graphs: reading the picture that shows where time goesjunior
- Sampling vs instrumentation profiling: why 99 Hz wins in productionmiddle
- Profile types: CPU, memory, off-CPU, mutex — which one to reach formiddle
- Continuous profiling: always-on flame graphs with eBPF and trace-id correlationmiddle
- How flame graphs are built from samples, and the production workflows that use themmiddle
- Linux perf, eBPF internals, PGO, and the limits of samplingsenior
- Profiling in production: security, war stories, OTel profiles, and the infrastructure designsenior
- The debugging funnel: SLO → RED → trace → profilejunior
- OTel architecture: one SDK, four signals, one wire formatmiddle
- Cost discipline: keeping observability under 5% of infra spendmiddle
- Scale, security, and the ROI of observable systemssenior
- Why profile first: measure where time actually goesjunior
- Amdahl''''s law and self-time: the ceiling on every speedup you can shipmiddle
- The measurement loop: microbench, macrobench, prod profile, observer effectmiddle
- Reading flame graphs: shapes, per-language profilers, and the 60-second scanmiddle
- Statistical baselines: why one run is not a measurementmiddle
- Profiler history and microbenchmark pitfalls: Knuth to GWPsenior
- Hardware counters, cold-start profiles, and profile securitysenior
- Continuous profiling at scale: costs, CI gates, trace correlation, and anti-patternssenior
- What makes a hot path: symptom vs causejunior
- Five shapes of hotspot: CPU, alloc, cache, lock, syscallmiddle
- Reading parent and child chains: where to apply the fixmiddle
- JIT deopt, the fix-and-verify loop, and PR-time profilingmiddle
- Hardware counters and Intel TMA: sub-category diagnosissenior
- False sharing and native-bridge hot pathssenior
- Hot paths in production: security, tail latency, and tooling lineagesenior
- Memory hierarchy: why the same O(N) loop can be 17x slowerjunior
- Row-major vs column-major: access order and the 9x gapjunior
- Branch prediction and branchless codemiddle
- Hardware prefetcher, TLB, and memory-level parallelismsenior
- GC basics: what the runtime taxes you forjunior
- GC algorithms: generational, concurrent, and per-runtimemiddle
- GC tradeoffs: pause, throughput, heap — and object poolingmiddle
- GC tuning: pacing, heap shape, and allocation observabilitymiddle
- GC internals: tri-color invariant, write barriers, and per-runtime deep-divessenior
- GC in production: observability, security, edge cases, and fleet governancesenior
- N+1: one logical operation, many round-tripsjunior
- Fix families: JOIN, IN, preload, and DataLoadermiddle
- Detecting N+1: query logs, APM traces, and CI gatesmiddle
- DataLoader: batching across resolver treesmiddle
- Cross-protocol N+1: HTTP fan-out and Redis MGETmiddle
- N+1 at scale: pool exhaustion, plan changes, and denormalisationsenior
- Batching: amortize fixed cost per operationjunior
- The batching window: size and wait timemiddle
- Batching in Kafka and Postgresmiddle
- io_uring and observability of batchingmiddle
- From Nagle to io_uring: evolution of batchingmiddle
- Backpressure, failure isolation, and batch security in productionsenior
- What a bundle actually costs: download, parse, compile, executejunior
- Core Web Vitals: LCP, INP, and CLSmiddle
- Code splitting: route-level, component-level, vendor splittingmiddle
- Tree shaking and compression: removing what you don''''t usemiddle
- Third-party scripts: the silent budget killermiddle
- CI enforcement and RUM: making budgets stickmiddle
- V8 JIT pipeline, HTTP priorities, and bundle securitysenior
- The performance loop: discipline, not a projectjunior
- Classify and fix: matching bottleneck families to remediesmiddle
- Observability stack and CI gates: catching regressions before they shipmiddle
- Incident to enforcement: SLO burn to verified fix in 35 minutesmiddle
- Culture, economics, and org-scale performancesenior