A Model Will Retry a Failing Tool Whether or Not That Is Safe

A retry is the most ordinary thing an autonomous system does with a failure, and the one a tool is least often built to survive. When a call comes back failed, the model may simply issue it again, on its own judgment, with less context than the engineer who wrote the tool and no way to know whether a second identical call is safe. That leaves the safety of the retry entirely to the tool. A tool that assumes a repeated call only arrives when repetition is harmless is making a bet the model never agreed to, and it loses that bet as a duplicate charge, a doubled record, or a run of failed calls that crowds out everything else in the context. Retry safety is not something the agent can enforce from outside. It is a property the tool either has or lacks, built in before the first failure or absent when it counts.

Handing the retry decision to the model also changes what a failure even is. A conventional caller tends to treat a tool result as binary: it worked, or it did not. An agent operating over real systems runs into a wider set of outcomes, and the two that matter most are the ones the binary view erases. A call can fail for a reason that a second attempt would clear, which means the failure carries a hidden instruction to try again. And a call can partly succeed, completing some of its work before stopping, which means the result is neither a success to build on nor a failure to discard. A tool that collapses both of these into a flat failure hands the model a lie of omission, and the model recovers from the lie by guessing. Designing for retry and partial success is the work of replacing those guesses with facts the model can read.

Worth retrying and safe to retry are two different questions

A retry has to clear two bars, and they are independent. The first is whether trying again could help at all: a failure caused by bad input will fail the same way however many times it runs, while one caused by a passing condition downstream may clear on the next attempt. Sorting failures along that axis, and stating the answer in the error rather than leaving the model to infer it, is the work of the error a tool returns when it fails. The second bar is the one that gets skipped. Even a retry that is clearly worth attempting can be unsafe to attempt, because a model that correctly reads a failure as transient and tries again can do real damage if the operation underneath was never built to be repeated. Worth retrying is decided by the error. Safe to retry is decided by the tool, and it is the harder half.

That second bar is what the rest of this comes down to, because it is where a well-reasoned retry turns into damage. There are three ways an operation earns the right to be retried: it absorbs a repeat without duplicating its effect, it reports a partial result instead of collapsing into an all-or-nothing failure, and it stops calling a dependency that has plainly stopped answering. Each corresponds to one of the failure modes a naive retry produces, and each is a decision made at the tool interface, long before any call fails.

Idempotency is what turns a retry from a gamble into a safe default

Start with the first way an operation earns a retry, which is idempotency. An operation is idempotent when calling it twice with the same input has the same effect as calling it once. Reads are idempotent by nature: fetching the same record ten times changes nothing. Writes that create or modify state are not, unless idempotency is built in deliberately, and this is the gap that turns a well-intentioned retry into damage. The model, seeing a transient failure, does the reasonable thing and tries again. If the first call actually succeeded and only its response was lost to a dropped connection, the second call runs the operation a second time, and now there are two records, two charges, two messages where there should be one. This is the most expensive of the retry failure modes precisely because the retry was correct. The model reasoned well and the system punished it, because the tool was not built to absorb a second attempt.

Making a state-changing operation safe to retry is a tool-interface decision, and there are two established ways to make it. The direct approach is to accept a caller-generated idempotency key with every state-changing request. The tool stores the key alongside the result of the operation, and if it ever sees the same key again it returns the stored result instead of executing a second time. The first call does the work; every repeat of it becomes a lookup that returns what the first call produced. The other approach is the conditional write: the tool inspects current state before applying a change and skips the change if it has already been applied, so reapplying a completed step becomes a no-op rather than a duplicate. Either turns a dangerous operation into one the model can retry freely, and the cost is modest, a little storage and a lookup per call. The benefit is that a network hiccup between the tool call and its response never becomes a double charge, and the model is free to treat transient failures the way it should, by simply trying again.

A batch that half-succeeds should never report as a failure

The binary view of tool results does its worst damage on operations that act on many items at once. A batch tool that processes a list will routinely complete some items and fail on others, and the shape of its response decides whether the model can recover intelligently or is forced into a bad choice. The wrong shape is a single generic error that says the batch failed, because that erases the work that succeeded. The model, told only that the operation failed, has two options and both are poor. It can give up on everything, discarding completed work. Or it can rerun the entire batch, which reprocesses the items that already finished and, if those operations are not idempotent, duplicates them. The generic failure has manufactured exactly the stale-state problem that careful retry design exists to prevent.

The right shape reports partial success as its own outcome and splits the result into two explicit lists: the items that completed, identified by their keys, and the items that failed, each carrying its own error and the context that explains it. Handed this structure, the model can do the only sensible thing, which is to act on the failures alone. It retries the items whose errors are transient, surfaces the ones that need a correction, skips the ones already done, and never touches the work that succeeded. Partial success is the mechanism that lets a long-running operation be resumed rather than restarted. Take it away and the model’s only tools for a half-finished batch are amnesia and repetition.

A dependency that keeps failing needs a circuit, not another attempt

Retrying is the right response to a transient failure, but a service that has genuinely fallen over does not present as a single transient failure. It presents as a stream of them, and a naive retry policy meets that stream by trying again, and again, converting one broken dependency into a flood of doomed calls. In an agentic system this flood has a specific and ugly cost: every failed attempt returns an error into the context window, and a model hammering a dead service buries its own working memory under identical rejections until there is no room left for the reasoning that would have told it to stop. The failure mode is the infinite retry loop, and it comes from the absence of two things, a limit on how many times to try and a delay between attempts, which is to say from treating a persistent outage as if it were a passing one.

The circuit breaker is the pattern that draws the line between the two. It sits between the tool and an unreliable downstream service and moves through three states. Closed is normal operation: calls pass through and failures are counted. When failures cross a threshold, the breaker opens, and calls are refused immediately without touching the downstream service at all, which stops the flood at once. After a set interval the breaker moves to half-open and lets a single probe call through. If the probe succeeds, the service has recovered and the breaker closes and normal traffic resumes. If it fails, the breaker returns to open and the interval resets, so the system waits again rather than resuming the barrage. The three parameters that make this work have to be set with intent. The failure threshold decides how many failures constitute an outage rather than normal variance; pitched too low, ordinary flakiness trips it, and pitched too high, it keeps feeding calls to a service that is already gone. The recovery interval decides how long to wait before testing again. And the fallback decides what the tool returns while the circuit is open, whether that is a cached value, a reduced response, or a clean error that tells the model to escalate. Without a fallback the breaker only converts failing calls into blocked ones, which stops the flood but leaves the model with nothing to act on.

Recoverability is a property of the interface, designed in or absent

These are not three separate techniques so much as three faces of one obligation. Idempotency makes an individual retry safe. Partial success makes a batch resumable rather than all-or-nothing. The circuit breaker makes a persistent outage a bounded event instead of a runaway loop. Each closes off one of the ways a reasonable retry decision turns into damage: the duplicate side effect when a non-idempotent call is repeated, the stale-state corruption when a half-finished batch is rerun, the loop that drowns the context when a dead service is retried without limit. They share a single premise, which is that in an agentic system the retry decision has left the engineer’s hands and moved to the model, and the model will make it whether or not the tool is ready.

That premise is what makes recoverability a design property rather than a runtime hope. A tool built without it does not announce the gap; it works in every test where nothing fails, because the gap only opens when a retry lands on an operation that could not survive it. The tool that has retry safety built in behaves the same on the happy path and diverges precisely when things go wrong, absorbing the second attempt, reporting the half-completed batch, refusing the call to a service that will not answer. The difference between the two is invisible until a failure exercises it, and by then the choice was made long ago, in whether the interface was designed to be retried or merely assumed it never would be. Failure is the common case in any system that touches the real world, so the interface that governs failure is exercised constantly, and building it as carefully as the success path is not defensive engineering. It is the ordinary cost of putting a tool in front of something that acts on its own judgment.