instrumentation.ts and tracing: one register() per process, one trace per request, and the sampling bill

register() runs once per process — everything else is a footgun

instrumentation.ts lives at the project root (next to app/, or inside src/) and exports one function with one guarantee: Next.js calls register() when a new server process bootstraps, before it serves any request. That guarantee is exactly what an OTel SDK needs — a tracer provider must exist before the first instrumented operation, and it must exist once. Every alternative violates one half or the other. Initializing at module scope in some lib/otel.ts imported from a layout re-executes per route bundle (each route compiles as its own entry on serverless) and double-registers providers. Worse, register() itself runs in both runtimes your app uses — and the Node SDK imports node:async_hooks and friends, which do not exist in the edge runtime. The canonical shape is a runtime guard with a dynamic import:

// instrumentation.ts — the only file Next.js promises to run once per server process
export async function register() {
  if (process.env.NEXT_RUNTIME === 'nodejs') {
    await import('./instrumentation.node'); // NodeSDK pulls in node:async_hooks — must never load in the edge bundle
  }
}

// instrumentation.node.ts — full control: exporter, resource, sampler
import { NodeSDK } from '@opentelemetry/sdk-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';

const sdk = new NodeSDK({
  serviceName: 'storefront',
  traceExporter: new OTLPTraceExporter({ url: process.env.OTLP_ENDPOINT }),
});
sdk.start();

If you do not need a custom pipeline, @vercel/otel collapses this to registerOTel({ serviceName: 'storefront' }) inside register() — it is runtime-aware and picks safe internals per runtime. Two honest caveats senior engineers trip on. First, “once per process” means once per cold start, not once per deployment: on serverless you have many concurrent processes, so anything stateful you build in register() — counters, in-memory sampling decisions — exists per instance, not globally. Second, the same file also exports onRequestError, a hook Next.js calls for server-side errors with the request context — the natural place to ship the error digests from the previous lesson to your error tracker, instead of scraping them out of stdout.

One request, one trace: built-in spans and propagation

With a tracer registered, Next.js emits spans for its own work — no code changes: a span per route render (named like render route (app) /dashboard), a span per fetch it instruments, spans for route handler execution, each carrying attributes such as the route template (NEXT_OTEL_VERBOSE=1 widens the set). The naming discipline matters more than it looks: a span must be named after the route template, never the concrete URL — /orders/[id], not /orders/9381 — because every distinct name is a new series in your backend, and a name-per-order turns a dashboard into a per-customer table.

Propagation is what turns three monitoring silos into one picture. The instrumented fetch injects the W3C traceparent header — 00-<trace-id>-<span-id>-<flags> — into outbound requests from server components, actions, and route handlers. Any backend running an OTel SDK in any language reads it and continues the same trace. The waterfall you see is then browser to RSC render to orders API to its database, one trace id end to end — and the on-call question “where did the 800 ms go” becomes a lookup, not a three-team Slack thread. If the backend shows up as a disconnected root span instead, the first suspect is a proxy or API gateway stripping traceparent on the way through.

Sampling and cardinality: the bill is a design constraint

Before you enable tracing, ask yourself: at your traffic rate, how many spans will fire per minute — and which ones will you ever query? The numbers from the Hook generalize. A span serializes to roughly 300–800 bytes with ordinary attributes; 31M spans/day is on the order of 10–25 GB/day ingested, and vendors price per GB or per million spans — either way, a span per data-cache HIT is paying to record that nothing happened. The knobs, in order of leverage. First, drop spans you will never query: filter cache-hit fetch spans and static-asset noise in a span processor or collector before they cost anything. Second, head sampling — ParentBasedSampler(TraceIdRatioBasedSampler(0.1)) — decides at the trace root and is nearly free, but it is blind: it drops error traces and slow traces at exactly the same rate as healthy ones, which is how the Hook team lost their incident. Third, tail sampling in a collector: buffer complete traces, then keep everything with an error, everything slower than a threshold, plus a few percent baseline. That is the policy you actually want, and its honest cost is operational — the collector holds every in-flight trace in memory for the decision window (commonly 10–30 s; at 600 rps that is thousands of buffered traces), and now you run and scale a collector. Together these three levers move from “pay for everything” to “pay only for what you would query”; skip the first two and you reach tail sampling with a dataset already too expensive to afford.

Cardinality is the quieter half of the bill. Attributes are indexed; put a user id, session id, or full URL into span attributes and your backend builds an index entry per user. The discipline that survives audits: route template, method, status, region — identifiers belong in logs (keyed by trace id), not in trace indexes. Runtime overhead, for honesty’s sake, is the cheapest part of the whole story: span creation costs single-digit microseconds of CPU and the BatchSpanProcessor exports off the hot path, so a typical app pays 1–3% CPU for tracing. The bill and the cardinality are where tracing actually hurts — which is why sampling strategy belongs in the design review, not in the panic after the first invoice.