AI / LLM Integration
Composing LLM apps: multiple-choice review
Crux Cross-track synthesis MCQs for the AI/LLM capstone — where caching, tools, RAG, streaming, budgets, agents, and evals fail at the seams between them.
Your altitude — climbing toward senior
ZeroJuniorMiddleSenior
You are at senior altitude — in orbitSix questions across the whole track. None tests a single layer — each puts two correct layers together and asks where the composition breaks. That seam is where production LLM apps actually fail.
Goal
Confirm you can reason about the seams: how RAG interacts with the cache, how a tool call interacts with the stream, how an agent loop interacts with a budget, and why evals must include the live retrieval path.
Quiz
Completed
A RAG-backed assistant has a long static system prompt marked with cache_control, yet its prod cache hit rate is near 0% — though it looked cached in the demo. What is the cause and the fix?
Heads-up TTL controls how long an entry survives, not whether the prefix matches. If the prefix differs every request, a longer-lived entry still never matches — the hit rate stays at zero. The problem is prefix instability, not eviction.
Heads-up Volume doesn't help when every request carries a different prefix. The demo 'cached' only because it repeated one identical question; real queries each embed different retrieved chunks inside the prefix.
Heads-up That trades the grounding feature for a cache metric — optimizing the wrong layer. The fix keeps retrieval and relocates it past the breakpoint so the static prefix stays cached.
Quiz
Completed
A streamed turn ends with stop_reason: tool_use partway through and the UI hangs on a spinner forever. Which mental model fixes it?
Heads-up Nothing errored — the model deliberately requested a tool. Restarting re-runs work and can loop; you must handle the tool_use transition and continue, not restart.
Heads-up max_tokens limits length, not the model's decision to call a tool. It stopped to request a tool; a higher limit doesn't change that branch.
Heads-up Buffering throws away streaming's latency benefit and still doesn't run the tool — you'd buffer, then still need to execute and resume. The tool-use branch is unhandled either way.
Quiz
Completed
Your agent has a token-budget dashboard with alerts, yet a bad input drove it into a retry loop that burned thousands of dollars overnight. Why didn't the budget protect you?
Heads-up Faster observability still only observes. By the time any alert fires the dollars are already spent; only a pre-call gate that refuses the next request actually bounds cost.
Heads-up A cheaper model makes a runaway loop cost less per step, but it still runs forever on bad input. The defect is the missing terminating condition, not the price per token.
Heads-up More retries is the opposite of the fix — it feeds the loop. The famous case ran 11 days for $47,000 precisely because nothing capped the iteration count.
Quiz
Completed
Your offline eval suite is green on every deploy, but users report worse answers right after a retriever change. Why did the evals miss the regression?
Heads-up Flakiness shows as noisy pass/fail, not consistent green hiding a real regression. This is a coverage gap: the retrieval seam is simply not in the test path.
Heads-up The change was the retriever, not the model. Re-running generation-only evals on frozen context still skips the path that actually regressed.
Heads-up The generation metrics can be fine in isolation; the problem is they run on frozen context. You need an end-to-end eval that includes live retrieval, not different generation metrics.
Quiz
Completed
A network blip truncates a streamed turn with stop_reason: tool_use but zero tool_use content blocks, and the agent goes idle silently. A later request is rejected with 'unexpected tool_use_id found in tool_result block'. What is the common root cause?
Heads-up Both are the same seam: the stream and the tool protocol share state across requests, and the composition never validated that pairing. A blank tool_use stop and an orphaned tool_use_id are two symptoms of unvalidated transitions.
Heads-up Neither symptom is hallucination — one is a truncated stream with no tool block, the other is a stale id from a crashed turn. The fix is protocol-level pairing and recovery, not prompt wording.
Heads-up HTTP idempotency dedupes retries; it does nothing about an empty tool_use stop or a tool_result referencing an id the model never emitted. You need state-machine tracking of tool_use ↔ tool_result.
Quiz
Completed
A RAG-backed agentic assistant ships: every component passed its own test, but the bill triples and answers feel slower in prod. What is the senior debugging move?
Heads-up They all still pass — that's the trap. The failure is between correct components (e.g. RAG busting the cached prefix while looping requests grow the transcript), so per-component tests can't surface it.
Heads-up The features have to coexist; the goal is a correct composition, not avoiding composition. Tracing the seam lets you keep all six layers and fix the boundary.
Heads-up More hardware pays for the broken seam instead of fixing it — the cache is still busted and the loop still grows the transcript. Cost and latency both come from the seam, not capacity.
Recap
The track’s through-line is one sentence: each layer is correct by its own contract, and the failure lives where one layer’s output silently violates the next layer’s input assumption. RAG changes the prefix the cache assumed was stable; a tool_use stop changes the stream the renderer assumed was prose; real input changes the loop the budget assumed would terminate; the retrieval path changes under an eval suite that assumed fixed context. You don’t debug these by re-testing pieces — you trace one real request end to end, and at every boundary ask what the next layer assumed that the last one just changed.