AI / LLM Integration
LLM cost budgets: multiple-choice review
Crux Multiple-choice synthesis across the LLM cost-budgets unit — token asymmetry, re-sent context, model routing, prompt caching, and in-process budget guardrails.
Your altitude — climbing toward senior
ZeroJuniorMiddleSenior
You are at senior altitude — in orbitSix questions that cut across the whole unit. Each mirrors a call you make on a real cost incident — not a definition to recite, but a tradeoff to weigh while the meter is running.
Goal
Confirm you can connect token pricing, re-sent context, routing, caching, and in-process budgets into one decision — the synthesis the overview lesson built toward.
Quiz
Completed
A support chatbot on Sonnet 4.6 ($3/M in, $15/M out) sends a 200-token question and gets a 1,500-token answer, mostly chain-of-thought the user never sees. Where is the spend, and what is the first lever?
Heads-up Even with a system prompt, output at $15/M against $3/M input is the expensive half here. A 1,500-token answer outweighs a 200-token question many times over; the verbose answer is the target, not the question.
Heads-up Output runs ~5x input on Sonnet ($15 vs $3 per M). A verbose answer is far pricier than a verbose prompt — that asymmetry is the whole point of the unit.
Heads-up Every generated token is billed at the output rate whether shown or not. Hidden chain-of-thought is paid output, which is exactly why a senior checks output volume first on a spike.
Quiz
Completed
A 50-turn chat re-sends a 4,000-token system prompt every turn, and the input bill is climbing. Which fix has the highest leverage?
Heads-up A cheaper model still re-sends the 4,000-token prefix every turn; you cut the rate but not the waste, and trade quality blindly on hard turns. Caching targets the unchanging re-sent input directly.
Heads-up That silences the only signal while the waste continues. Caps are a backstop, not an optimisation — removing the alert is how a runaway goes unnoticed until the invoice.
Heads-up History trimming is a real but separate lever; it does nothing for a fixed system prefix re-sent every turn, and aggressive truncation breaks multi-turn coherence. Caching is the targeted fix here.
Quiz
Completed
A team routes the easy 80% of requests to Haiku ($1/$5) and escalates failures to Opus ($5/$25). After launch the bill barely moved. Most likely cause?
Heads-up Routing can cost more than not routing when the cheap-then-expensive path fires too often. The classification step plus the escalation retry add cost; the savings are conditional on a low escalation rate.
Heads-up Haiku is $1/$5 and Opus $5/$25 — a 5x gap on the slice routed cheap. The pricing gap is real; the problem is that escalations are eating the savings.
Heads-up Running the expensive model to decide which model to run defeats the purpose. A heavy router can cost more than it saves; keep the routing decision cheap or heuristic.
Quiz
Completed
An autonomous agent loop with no iteration cap runs overnight and bills $4,300. Why didn't the $1,000/month spend cap stop it?
Heads-up Spend caps count total spend, not one token class. The gap is timing and granularity — a monthly total can't catch a per-second runaway.
Heads-up Even on the covered key, the monthly cap fires only after the cycle total crosses the threshold — far too late for a loop spending thousands in hours.
Heads-up Caching is unrelated to cap enforcement. The real fix is an in-process kill switch on cost velocity and iteration count, not a billing-dashboard limit.
Quiz
Completed
Why does an uncapped agent loop cost superlinearly in the number of iterations, not just linearly?
Heads-up Calls are not the same size. A stateless model re-sends the growing transcript, so each later call is bigger than the last; the total is a sum of a growing series, not a flat per-call cost.
Heads-up The superlinear growth comes from re-sent input context accumulating, not from output exploding. Each turn re-pays for all prior turns' content.
Heads-up Per-token rates are flat; many providers give volume discounts, not surcharges. The superlinearity is structural — accumulating re-sent context — not a pricing tier.
Quiz
Completed
You are designing cost controls for an LLM feature. Which ordering — from cheapest first-line defense to last-resort — reflects the unit's priority?
Heads-up The kill switch is the last-resort backstop, not the first line. You reduce normal-path spend (routing, caching, trimming) before leaning on a circuit breaker that only fires in a runaway.
Heads-up Raising the cap is not a control at all — it loosens the backstop. Caching and routing reduce spend; the cap only bounds the worst case after the fact.
Heads-up A global downgrade ignores re-sent context and trades quality blindly. Routing must be selective, and caching plus in-process budgets still matter — a single lever doesn't bound a runaway.
Recap
The through-line is one decision tree: output costs ~5x input so cap it first; the stateless model re-sends the system prompt, history, and RAG every turn so cache the stable prefix and trim the volatile parts; route the easy majority cheap and watch the escalation rate; and because a runaway loop costs superlinearly while a monthly cap is measured in days, the real brake is an in-process budget plus a kill switch on cost velocity. Every control reduces or bounds re-sent context and output before it bounds the bill.