Async context per language, service mesh, B3 migration, and security

A Java service migrates from thread-pool-based request handling to Project Loom virtual threads. After the migration, 30% of traces are orphans. The OTel SDK version didn’t change. What broke?

Async context propagation: language by language

Each runtime has its own mechanism for “what is the current execution context?” — and OTel hooks into that mechanism. When you cross a primitive the runtime doesn’t carry context through automatically, context is silently lost.

Node.js: AsyncLocalStorage (built into Node 12+) is the substrate. OTel hooks into it for in-process propagation. Pitfalls: setTimeout, setImmediate, queueMicrotask, and any third-party promise wrapping that creates a new AsyncLocalStorage context can lose trace context. OTel auto-instrumentation patches the common ones, but custom libraries break it. Fix: context.bind(ctx, fn) before passing a callback to any async boundary.

Python: contextvars from PEP 567 is the substrate; works automatically for asyncio but not threading without ContextVar.copy(). If you spawn threads manually, the OTel context from the parent thread is not inherited. Fix: pass the current context to the child thread and set it via context.attach(ctx) on entry.

Java: traditional ThreadLocal plus a Span.makeCurrent() try-with-resources block. Project Loom virtual threads are mostly transparent for simple cases, but require careful Scope nesting — if a virtual thread is created inside a Scope, the scope must outlive the virtual thread, otherwise the context is closed while the thread is still running. Fix: structure virtual-thread creation to happen inside the scope, not outside it.

Go: explicit context.Context plumbing — every function takes ctx, every span lives in ctx. Go made the right architectural choice early; context never flows implicitly. The failure mode in Go is not losing context accidentally, but forgetting to thread ctx through a function call chain. Fix: pass ctx everywhere; use go vet and staticcheck to flag missing context parameters.

Browsers: zone.js (Angular’s solution for patching async primitives), or the TC39 AsyncContext proposal; OTel-JS supports both via plugins. Service workers and Web Workers require explicit context pass-through.

B3 vs W3C: the migration story and safe sequence

Before W3C Trace Context, Twitter’s Zipkin/Brave used B3, with two variants: B3 multi-header (X-B3-TraceId, X-B3-SpanId, X-B3-Sampled as separate headers) and B3 single-header (all three combined in one). B3’s original trace-id was 64-bit; it was extended to 128-bit for W3C compatibility.

Safe migration sequence:

1. Audit: identify every service still emitting B3-only headers.
2. Deploy composite propagator everywhere: register W3C TraceContext + B3 multi + B3 single at every service. Write outbound W3C only; extract inbound from both. This is the “read-both, write-W3C” phase.
3. Verify: confirm orphan-span rate doesn’t regress.
4. Phase out B3 outbound: after downstream services are confirmed to read W3C, disable B3 outbound at each upstream.
5. Remove B3 extractor: after a quarter at zero B3 inbound, replace the composite propagator with W3C-only.

What goes wrong if steps are skipped:

• Skipping step 2: upstream sends W3C while downstream only reads B3 → traces split.
• Skipping step 3: a propagation regression goes silent for weeks.
• Skipping step 5: double header bytes per request indefinitely.

Trace context across service mesh

Envoy, Linkerd, Istio, Cilium data planes participate in tracing two ways:

1. Pass-through (always): the sidecar forwards traceparent/tracestate/baggage headers on every HTTP and gRPC request transparently.
2. Emit their own spans (optional but recommended): when enabled, the sidecar creates a span for the network hop, showing sidecar latency, connection pooling, and TLS handshake timing distinct from application latency.

Configuration: the mesh proxy needs the tracing-collector address and the sampling decision (usually inherit the incoming flag). The mesh’s sampling decision must agree with the application’s; mismatched rates produce inconsistent traces.

The limit: service mesh only handles HTTP and gRPC. Queue consumers, timers, and fire-and-forget callbacks still require explicit application-level propagation. The mesh is not a substitute for OTel SDK instrumentation; it adds a network-hop span, it does not replace application spans.

Security: the trace-id as a tracking identifier

Trace-ids are unique per request, 128 bits of entropy, propagated in HTTP headers and visible to anyone who can inspect traffic between the client and the origin. This makes them powerful debugging tools and equally powerful potential trackers.

The risk: if a third party (a CDN, a marketing pixel, a CSP-allowed analytics service) can read the traceparent header from the user’s outbound requests, it can correlate user activity across sites that share the same tracing infrastructure.

Mitigations:

• The W3C spec recommends that user-facing services do not propagate traceparent in responses (responses are to the user, not part of an upstream call).
• Browser-side OTel SDKs should limit propagation to same-origin and explicitly-allowed CORS origins (TraceContextPropagator.allowedOrigins list).
• Production teams maintain an allowlist of downstream hostnames that receive traceparent and audit it quarterly.
• Baggage applies identically — anything in baggage is observable by every downstream including third parties.