Observability
OTel: multiple-choice review
Crux Multiple-choice synthesis across the OTel unit — the four-piece architecture, signals and OTLP, Collector pipeline, sampling, vendor-neutrality, and operations.
Your altitude — climbing toward senior
ZeroJuniorMiddleSenior
You are at senior altitude — in orbitSix questions that cut across the whole unit. Each mirrors a real architecture or incident decision — not a definition to recite, but where to make a change and why it lands there.
Goal
Confirm you can connect the four pieces — API, SDK, Collector, OTLP — to the decisions they govern: where portability lives, what makes signals correlate, how sampling and the Collector pipeline compose, and how the whole thing fails in production.
Quiz
Completed
A platform team claims full OTel vendor-neutrality. Two facts surface in review: application code imports the vendor's dd-trace SDK directly, and a Collector sits in front of every service converting to OTLP. Are they neutral, and why?
Heads-up The Collector can reshape data, but the migration cost was already incurred upstream: every service imports the vendor SDK. You would rewrite instrumentation in every service to actually move, so the edge — not the Collector — sets neutrality.
Heads-up A Collector is necessary infrastructure but not sufficient for neutrality. If the application edge is not OTLP, the lock-in lives in application code regardless of what the Collector does.
Heads-up Vendor distros that accept and emit OTLP are acceptable; the silent erosion is vendor SDKs in app code and proprietary Collector-only processors, not the distro itself.
Quiz
Completed
A team needs tail sampling that keeps 100% of error traces. They run a single agent Collector as a DaemonSet (one per node) exporting straight to the backend. Why does this not work, and what is the minimal fix?
Heads-up The processor binary is the same; the problem is data locality. An agent only receives the local node's spans, so it never has the full trace context tail sampling requires — not a missing feature.
Heads-up Head sampling decides at root-span start, before any error has occurred — it is blind to errors. Only tail sampling, seeing the completed trace, can keep 100% of error traces.
Heads-up Buffer size is irrelevant when the agent never receives the other nodes' spans of a trace. The issue is that the full trace is not co-located, not that the buffer is too small.
Quiz
Completed
Two services emit HTTP telemetry. Service A tags the route as http.route; service B tags it as http_route. OTLP export succeeds for both and no SDK error fires. What actually breaks, and where?
Heads-up OTLP places no constraints on attribute names; both export fine. Semantic Conventions are a naming agreement enforced by governance, not by the wire protocol.
Heads-up Neither the SDK nor the Collector enforces Semantic Conventions at write time. That is exactly why the drift is silent and why a platform wrapper plus CI lint is the only durable fix.
Heads-up transform/attributes processors can rename fields, but only if you write a rule for it — there is no automatic correction. By default the non-standard name flows straight through.
Quiz
Completed
In a production traces pipeline you have memory_limiter, batch, and tail_sampling. Why does memory_limiter belong first, and what is the consequence of putting tail_sampling first?
Heads-up The Collector runs processors in the exact order you declare them. Ordering is a correctness and protection decision, not a hint.
Heads-up tail_sampling does not reduce in-flight memory — it buffers every span for 30-60s before deciding. Running it before memory_limiter means the limiter cannot protect against the very buffering that causes the OOM.
Heads-up memory_limiter checks process RAM independently of batching and must come first so it can refuse new records before the costlier processors consume more memory.
Quiz
Completed
A metrics backend's cardinality explodes after a fleet rollout of OTel auto-instrumentation. The new dimension driving it is the HTTP client's url.full attribute. What is the root cause and the correct fix?
Heads-up Scaling storage treats the symptom and the cost keeps climbing. The cause is an unbounded-cardinality attribute; the durable fix is to template or drop it, not to absorb infinite series.
Heads-up Tail sampling acts on traces, not on metric series cardinality. The exploding dimension is a per-request attribute on metrics — url.full — independent of sampling.
Heads-up Histogram type controls latency-bucket resolution, not label cardinality. The cardinality is in the url.full label dimension, which neither histogram type changes.
Quiz
Completed
Your single OTel Collector gateway crashes during an incident. The on-call dashboards stay green and no alerts fire, yet user-facing errors are spiking. What is the lesson, and the structural fix?
Heads-up The dashboards are green because their data source — the Collector — is down, not because the system is healthy. Absence of telemetry is being read as absence of problems.
Heads-up More logging still flows through the same failed Collector. The fix is making the Collector itself highly available and self-monitored, not adding upstream volume to a broken pipe.
Heads-up Direct export loses buffering, central policy, and uniform processing, and creates a new single point of failure per service. The answer is to make the Collector tier HA and observable, not to remove it.
Recap
The through-line: portability is decided at the application edge (emit OTLP there or nowhere), correlation is decided by Semantic Conventions (consistent names or invisible services), and the Collector is where policy lives — processor order protects it, sampling composes head-cheap with tail-curated, and the gateway must be HA and self-monitored because when the Collector fails, your visibility fails silently. Every production failure in this unit — eroded neutrality, split traces, naming drift, OOM, cardinality leaks, blind dashboards — resolves back to one of those four pieces.