Error Recovery

Error recovery is the layer of an agent harness that decides what to do when a model call comes back wrong. It classifies the failure, repairs whatever caused it — waiting out a rate limit, compacting an oversized context, raising the output budget, switching model — and only then retries. A retry without a repair is just the same call again.

Think of a sofa and a doorway. Sometimes the door sticks and a second shove opens it. Sometimes the sofa is genuinely wider than the frame, and you can shove until your shoulder gives out; the only thing that works is taking the legs off. And sometimes you are at the wrong building entirely, in which case every shove is a donation to the void. Retry logic that cannot tell these three apart will do the wrong one with total confidence, five times, with exponential pauses in between.

How Error Recovery Works

Between one attempt and the next, exactly three things can change: the clock, the request, or nothing. That is the whole taxonomy, and every rung of the recovery ladder is one of those three.

The Clock: Errors a Wait Repairs

This fixture’s 529 overloaded represents temporary service pressure. A real 429 rate_limit often does too, but it can also indicate quota or other limits that waiting alone will not resolve. Nothing is wrong with what you sent. The world will be in a different state in a second, so resending the identical bytes after a wait is not laziness — it is the correct repair. This is the one case where blind retry is genuinely right, which is exactly why blind retry survives in so many codebases.

The Request: Errors Only a Rebuild Repairs

A 400 context_length_exceeded says your prompt plus the output you reserved does not fit the model's window. Waiting does not shrink a prompt. The repair is to rebuild the request smaller: summarize the old turns, keep the recent ones verbatim, resend. A response that stops with stop_reason: max_tokens is the mirror image — the context was fine, the answer needed more room than you allowed, so you double max_tokens. Repeated 529 responses can trigger a policy to try an authorized, compatible backup model. They do not prove that the original model was the wrong choice or that the outage will never clear.

Nothing: Errors No Retry Repairs

A 400 invalid_request_error means the request itself is malformed — a tool schema that is not valid JSON Schema, say. Waiting, compaction, or a blind fallback does not repair that malformed schema. Correcting the request can, but this fixture stops and surfaces it for that repair. The correct recovery is to stop on attempt 1 and surface the error. Stopping is a repair decision, not the absence of one.

Where the Classification Comes From

The animation shows classify(response) as one call, which hides the part that actually trips people: the evidence lives in three different places. The HTTP status carries the transport-level failures (429, 529, 400). The error body carries its own discriminator — error.type in the Anthropic-style response shape used here, with values like rate_limit_error, overloaded_error, and invalid_request_error — but it does not separate every pair of errors that share a status code. An oversized prompt and a malformed tool schema both arrive as 400 invalid_request_error; this fixture's context_length_exceeded is a teaching label, and a real harness tells those two 400s apart by the error message. And a truncated answer is not in an error body at all: it is stop_reason: "max_tokens" on a response that succeeded.

That last one is the trap worth naming early: ok is something your harness decides, not something the transport tells you. A truncated answer arrives as HTTP 200. If your retry logic only catches exceptions, it will never see that failure at all.

Error Recovery Step by Step

The default run puts a long-session request against a model whose window it no longer fits, and hands it to a harness that classifies before it retries. Eight beats, start to finish.

1. The request is assembled. Model primary, 203,800 prompt tokens, max_tokens 4,096, 34 messages. The model's context window is 200,000.
2. Attempt 1 goes out. All 203,800 tokens of it.
3. It fails: 400 context_length_exceeded. The server checks prompt plus reserved output against the window — 203,800 + 4,096 = 207,896 against 200,000. Here the prompt alone already overruns it; the reservation only decides the verdict when you are sitting near the edge, and then it is the thing that pushes you over. Note what did not happen: nobody timed out, nothing was overloaded. This request was rejected before a single token was generated, and it will be rejected the same way forever.
4. The harness classifies it: context_too_long. This is the beat the whole lesson turns on. Resending is not on the table, because the response is a deterministic function of what was sent, and what was sent has not changed.
5. The repair runs: compact. The 28 oldest messages (172,300 tokens) collapse into a single 1,450-token summary; the last 6 messages (18,900 tokens) survive verbatim. Add the 12,600 tokens of system prompt and tool definitions, which this harness preserves unchanged, and the request is now 32,950 prompt tokens across 7 messages. Same task and recent tail, about 84% fewer prompt tokens; the summary is lossy.
6. Attempt 2 goes out. 32,950 tokens, same model, same max_tokens. The prompt content and message count differ from attempt 1. That is the only reason this attempt can end differently.
7. It returns ok. 32,950 + 4,096 = 37,046, comfortably inside the window.
8. The bill: 2 attempts, 0.0 seconds of waiting, 236,750 prompt tokens offered. Zero waiting is not a rounding artifact. A deterministic error has nothing to wait for, so the harness does not back off before a repair — it backs off only when the clock is the thing being repaired.

Now flip the policy knob to blind and run the same failure. The harness sleeps 1 second and resends 203,800 tokens. Same error. Sleeps 2 seconds, resends 203,800 tokens. Same error. It does this five times: 5 attempts, 31 seconds of backoff, 1,019,000 prompt tokens offered, no answer. The attempt ledger fills with five byte-identical rows, which is the most honest picture of blind retry anyone has ever drawn.

Why Blind Retry Costs More Than It Looks

Run the grid and the pattern is not "classification good, blind retry bad." It is narrower and more useful than that.

On a transient overload, the two policies are indistinguishable: both succeed on attempt 2 after the same 1-second wait, having sent the same 49,200 tokens. Classification bought nothing. If every error your agent ever saw were a rate limit, blind retry would be the right engineering call and the ladder would be overhead.

The divergence starts the moment an error is not about timing. On the context overflow above, blind retry spends 4.3× the prompt tokens and 31 seconds to reach the identical failure. On an invalid request, it spends 5 attempts and 31 seconds to learn what attempt 1 already said in plain text. Classification reaches the same failure in 1 attempt and 0 seconds — the run still fails, using one-fifth as many attempts and offered prompt tokens, with the real error at the top of the log instead of buried under four retries of noise.

One honest correction to that 4.3×, because the lesson's own ledger flatters it: compaction is not free. Producing the 1,450-token summary means a model has to read the 172,300 tokens being summarized, and this harness charges nothing for that. Adding its 172,300-token input gives roughly 409,000 input-token exposures rather than 236,750 — a 2.5× ratio to blind’s offered tokens instead of 4.3×. This is still not a price estimate: rejected requests, caching, summary output, and model rates affect billing. The direction is unchanged and the outcome is not close (one run answers, the other never does), but a repair that rewrites the request usually costs a call of its own, and a retry budget that ignores that will under-count.

That is the asymmetry worth internalizing. Classifying costs you one if statement. Not classifying costs you the full retry budget on every failure that a retry cannot fix, in these deterministic non-transient fixtures; this run does not measure real error frequencies.

There is one more subtlety the grid exposes. A 529 that clears and a 529 from a model that is simply down are the same response. The harness cannot distinguish them by reading, only by trying: it waits 1 second, waits 2 seconds, and when the third attempt still fails it stops treating the error as transient and switches to the backup model, which answers on attempt 4. The retry budget is not a safety limit here. It defines when the harness stops waiting and tries another route; it is not proof that the original model cannot recover.

Edge Cases

• Compaction that does not compact enough. If the recent messages you keep verbatim already exceed the window, one pass of compaction leaves the request still too large. A real harness either compacts harder (keep fewer recent turns) or fails with a clear message; a harness that loops on the same repair has reinvented blind retry at a higher altitude.
• A backup model that is also down. Fallback assumes somewhere else is healthy. When it is not, you exhaust the budget anyway — and you now have two providers' worth of latency in the bill. Fallback ladders need a floor.
• Retrying a non-idempotent call. Everything above assumes resending is free. If the call already ran a tool that charged a card or sent an email, a retry is not a repair; it is a second charge. Use stable idempotency keys where the operation supports and enforces them, or another deduplication/reconciliation protocol. A key alone provides no guarantee if the receiver ignores it.
• A truncated answer that is truncated again. Doubling max_tokens works until it hits the model's own output cap. Past that, the fix is not a bigger budget but a smaller task.
• Backoff with no jitter. Every client that failed at the same instant wakes at the same instant. The lesson's 1-2-4-8-16 schedule is deliberately naive so the arithmetic stays legible; production adds randomness so the retry storm does not re-form.

Common Mistakes

• Treating "it failed" as a single condition. One except block and one sleep applies the clock repair to errors that have nothing to do with the clock. The error object already carries the information you need; read it.
• Sleeping before a repair. If you compact the context and then wait 4 seconds, you paid for a delay that repaired nothing. Backoff belongs on the transient branch only.
• Sleeping after the final attempt. The naive loop in this lesson's blind variant sleeps 16 seconds after failure number five and then raises anyway. That last sleep is pure cost; it is also extremely common.
• Catching only exceptions. A truncated answer is an HTTP 200. If your definition of success is "no exception was thrown," you will hand a half-finished answer to the next step of the agent and never know.
• Retrying a malformed request unchanged. This schema error needs a corrected request. The context-overflow preset also uses 400, so status alone cannot decide whether repair and retry are appropriate. Retrying it converts a clear, immediate bug report into a slow, confusing one.
• Letting the retry budget hide a real outage. If your ladder silently falls back to a backup model on every call, you have stopped noticing that the primary is down. Recovery should be invisible to the user and extremely visible in your logs.

A Note on Simplification

This lesson runs a deterministic teaching harness, not a production client. The request sizes, the compaction ratio, and the failure world are authored so the arithmetic stays checkable on screen; no model is called, and nothing here is a recording of one.

The sharpest omission is the one above: the repairs are priced at zero. Compaction bills no tokens and no latency here, when in reality it is a model call over the history being summarized; a fallback bills nothing for the second provider's cold start. Both make the classifying run look cheaper than it is, which is worth knowing before you quote a ratio from this page.

Real recovery layers carry more machinery this animation leaves out: jitter on the backoff, honoring the server's retry-after header instead of guessing, resuming a stream that dropped mid-answer, idempotency keys so a retried call cannot double-charge, circuit breakers that stop hammering a provider that is clearly down, per-error-class budgets rather than one shared counter, and token-bucket rate limiting on the client side so many requests never fire. The taxonomy is real and the ladder is real; treat the numbers as a worked example rather than as production defaults.