Read real agent-loop snippets — tool dispatch, step caps, context trimming, error recovery — predict the behaviour, and pick the highest-leverage fix.
AISenior◷ 14 min
Level
FoundationsJuniorMiddleSenior
The agent loop is where the money is spent and the bugs hide. Read each snippet the way you would read it in a code review on a service that bills per token, then choose the fix a senior makes first.
Goal
Practise the loop you run on every agent: read the control flow, predict where it runs forever, overflows, or thrashes, and reach for the structural guard before blaming the model.
Snippet 1 — the loop with one exit
def run_agent(task, tools): messages = [SYSTEM, {"role": "user", "content": task}] while True: resp = model(messages, tools) # THINK if not resp.tool_calls: return resp.content # only exit: model stops calling tools messages.append(resp) for call in resp.tool_calls: result = dispatch(call, tools) # ACT messages.append(result) # OBSERVE
Quiz
Completed
This loop is correct ReAct but unsafe for production. What is the single most important guard to add, and why that one first?
Heads-up Error handling matters, but it is secondary — and feeding errors back without a cap can itself cause an infinite retry. The first defect is that nothing bounds the number of iterations at all.
Heads-up Caching shaves cost but does not stop a runaway loop; an agent thrashing on new-but-useless states still runs forever. You need a hard bound on iterations and spend.
Heads-up Streaming changes how output arrives, not whether the loop terminates. The missing piece is a termination guard, not a transport change.
Snippet 2 — the step cap that hides a bug
for step in range(MAX_STEPS): # MAX_STEPS = 100 resp = model(messages, tools) if not resp.tool_calls: return resp.content messages.append(resp) for call in resp.tool_calls: messages.append(dispatch(call, tools))# fell out of the loop: cap hitreturn "Sorry, I couldn't complete that."
Quiz
Completed
Telemetry shows ~30% of runs fall through and return the apology. The cap is doing its job — so what does a senior conclude?
Heads-up Raising the cap lets a quadratic loop burn far more tokens before tripping, and a loop that needs hundreds of steps has usually lost the plot. The fix is the underlying failure, not more rope.
Heads-up Returning partial work can be reasonable, but it doesn't address why a third of runs can't finish. The cap firing that often points to a structural defect upstream.
Heads-up A 30% cap-hit rate is a loud signal of thrashing or a broken tool, not a baseline to accept. The cap masks it; diagnosis from traces is required.
Snippet 3 — context trimming
def trim(messages, budget=8000): # keep most recent messages until we're under the token budget kept = [] total = 0 for m in reversed(messages): total += count_tokens(m) if total > budget: break kept.append(m) return list(reversed(kept))
Quiz
Completed
This keeps the loop under the window, but it has a failure mode that surfaces on long tasks. What is it, and the fix?
Heads-up Approximate counting is fine with a safety margin; that is not the structural bug. The real defect is that the unconditional 'keep most recent' rule evicts the instructions the task depends on.
Heads-up Two reversals are O(n) and negligible next to model calls. The substantive bug is what gets evicted — the pinned context — not the list mechanics.
Heads-up Keeping only the oldest would discard recent tool results the model needs to act now. The right answer is to pin the instructions and trim/summarise the middle, not to flip which end you keep.
Snippet 4 — error recovery
for step in range(MAX_STEPS): resp = model(messages, tools) if not resp.tool_calls: return resp.content messages.append(resp) for call in resp.tool_calls: try: result = dispatch(call, tools) except ToolError as e: result = {"role": "tool", "content": f"Error: {e}"} # feed error back messages.append(result)
Quiz
Completed
Feeding the error back lets the model adapt — but a load test shows runs where the same tool fails identically dozens of times. What guard closes the gap?
Heads-up Catching more exceptions makes the loop more resilient to crashing but more prone to thrashing — it keeps feeding failures back forever. The missing piece is a cap on identical/failed retries.
Heads-up Backoff helps for transient/rate-limit errors, but for a deterministic failure (bad args, not-found) it just delays the same identical failure. You still need a dedup/retry cap at the loop level.
Heads-up Crashing throws away the model's ability to recover from recoverable errors. The goal is bounded recovery — feed back, but cap identical retries — not no recovery.
Recap
Every agent incident is read in the loop: a single ‘model stops’ exit is unsafe, so add a hard step cap plus a wall-clock/token budget; a cap that fires routinely is a seatbelt catching a real bug, not a fix to loosen; naive context trimming evicts the pinned system/task messages and makes the agent forget its job, so pin instructions and summarise the middle; and error-feedback without a per-tool retry/dedup cap turns recovery into an infinite loop of valid calls. Read the control flow, find the unbounded path, add the structural guard, then re-run under load to confirm.
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.