Every Field a Tool Returns Is a Cost the Model Keeps Paying

The most-watched numbers in an agent system are usually the prompt and the model’s own output. The instructions get versioned, the responses get graded, the system prompt gets audited line by line. Meanwhile the largest and least-examined source of tokens in a long-running agent is often the raw output of the tools it calls. A single API response can be an order of magnitude larger than the prompt that triggered it, and almost none of that volume is chosen deliberately. It is whatever the underlying service happened to return.

That gap between what a tool returns and what the agent actually needs is not a cosmetic problem. Every field that lands in context is something the model has to read and reason around on this step and every step after it. The token bill is only the visible part. Underneath it, verbose output dilutes the model’s attention and buries the signal you care about, lowering the ceiling on accuracy in a way that looks, from the outside, like the model simply got worse at its job. The fix is architectural, it belongs in a specific place, and most teams put it in the wrong one.

The output a tool returns and the input an agent needs are two different sets

Tools are built for general consumption. A well-designed endpoint returns a complete, self-describing object: the fields you asked about, wrapped in everything the service maintainers thought some caller might someday want. Provenance and versioning stamps, paging machinery, rate-limit headers, identifiers for records you never asked to join against. That generosity is correct for an API. A human reading documentation benefits from seeing the full shape of a response, and a downstream program can ignore the fields it does not use at zero cost.

An agent is not a downstream program that ignores fields at zero cost. When a tool result enters the context window, the model does not have the option of skipping past the irrelevant parts for free. Everything in context is read. A response with fifty fields where the task depends on three has not delivered three useful fields and forty-seven inert ones. It has delivered three useful fields suspended in forty-seven fields of noise, and the model now has to locate the signal inside that noise every time it reasons over the accumulated context.

The useful framing here is signal-to-noise ratio applied to the context window itself. The window is a shared, finite working memory, and everything competing for space in it is either content that moves the current task forward or content that does not. Raw tool output tends to be mostly the latter. It lowers the ratio of relevant to irrelevant material, and it does so with nothing to flag it. The call succeeds, the data is correct, the agent keeps going, and the answer quietly gets worse. There is no error to catch and no log line to read, which is exactly why the problem survives so long in production.

The larger cost is misdirected attention

The intuitive cost of bloat is the token count, and that cost is real, but it is the smaller of the two. The larger cost is what verbose output does to where the model looks.

Position is not neutral inside a context window. The same fact is worth more to the model at either edge of the window than it is buried in the center, a property well enough established to have earned a name, the lost-in-the-middle effect. This is not a defect awaiting a patch in some later model. It is how attention distributes over a long sequence, and it carries a direct consequence for anyone building agents: when unreduced tool payloads accumulate, the fields the task actually depends on get displaced toward the middle, hemmed in by metadata no step will ever read, precisely the region the model weights least.

What that produces is a failure wearing a disguise. It does not present as a fault in the tool layer, where it originates. It presents as bad judgment. The model returns a wrong answer while its reasoning, traced step by step, holds up cleanly. It understood the task and followed the instruction. The load-bearing field was simply parked where the model’s attention runs thin, drowned out by the volume around it. You can lose hours auditing a prompt for a defect the prompt never contained. The information was present the whole time. It was just placed where the model could not give it the weight it deserved.

Bloat compounds with the length of the task

Measured against a single call, bloat looks trivial. One tool invocation returns a padded object, the model answers, and the excess is confined to that one exchange with a bounded impact. That is the view from which it never seems worth the effort to fix, and for one call in isolation the view is correct.

Agents do not work one call at a time. Their defining behavior, the thing that separates an agent from a single model invocation, is that they act, read the result, and choose the next action, turn after turn. Every turn writes its full payload into the same window, and the window does not clear between turns. So the padding is cumulative in the most literal sense. A field retrieved early is still physically sitting in the context many calls later, only now it is layered beneath everything deposited since, and the step that finally has to reach back for it has to locate it under all of that. The deeper into a session the agent gets, the more accumulated clutter stands between it and any earlier fact it needs.

This is the shift that changes bloat from a minor inefficiency into a genuine reliability problem. The cost is not additive, it is compounding. Every step you add to the workflow multiplies the accumulated noise the model has to read past, and the degradation grows with the length of the task. Worse, it grows exactly along the axis where you most need the agent to hold up. Short workflows tolerate a lot of sloppiness. Long ones are where reliability matters most, and long ones are exactly where untrimmed output hurts worst. An agent that behaves in a three-step demo and unravels on a fifteen-step real task is frequently not hitting a limit of its reasoning. It is drowning in its own tool history.

The waste is recurring, charged again on every downstream request

The economics of tool bloat are frequently misunderstood as a one-time cost, as if the tokens are spent once when the response arrives. They are not. Because context accumulates and is re-sent, the tokens a bloated response adds are paid for again on every subsequent request in that session.

Consider what happens after a verbose result enters the window. The next model call re-processes it. So does the call after that, and every call until the conversation ends or the context is actively pruned. A payload of irrelevant fields is not a single charge, it is a subscription. Multiply a modest amount of per-call waste by the number of calls in a long session and the total is not a rounding error, it is a meaningful and entirely avoidable line item, one that scales with exactly the long-horizon workloads that are most expensive to begin with.

Latency behaves the same way. A larger context takes longer to process and longer to generate from, and that tax is paid on every request regardless of whether the extra tokens changed the answer. Filler you accept once slows down every response that follows it.

This is worth dwelling on because it inverts the usual shape of a performance decision. Most tuning forces a trade: pay more to go faster, or accept a worse answer to spend less. Trimming tool output has no such tension, and the reason is mechanical. Cost, latency, and accuracy were all being dragged down by the identical thing, the fields the model never needed, so pulling those fields out repairs all three in a single move. You are not balancing one currency against another. You are removing a common cause.

Trimming at the display layer fixes nothing that matters

Here is where teams that have recognized the problem most often misplace the solution, and it is worth being precise about, because the mistake feels like progress while it is happening.

The natural instinct, once someone notices the clutter, is to tidy what people see. Build a view that surfaces the handful of fields a reviewer cares about and suppresses the rest. The screen gets clean, the team exhales, and the bloat appears to be gone. It is not gone. It has only been hidden from the one party that was never suffering from it.

The reason is a question of order, not of tidiness. Trace the data flow: a tool returns its payload, that payload enters the model’s context, the model reasons over it, and only then does anything render for a human. The interface sits at the end of that chain. A trim applied there acts on a copy the model has already read in full, so it cannot retroactively unsee the fields it was handed. The agent goes on reasoning over every suppressed value on every subsequent call, and the accuracy that the clutter was quietly eroding keeps eroding. A spotless dashboard resting on a context window packed with raw payloads has solved a problem no one had and left the real one untouched.

Reduction only helps the model if it happens before the boundary the model reads, at the point where the tool result is received and adapted, ahead of context rather than behind it. Where the trim sits in the pipeline is the whole of the matter; how polished the eventual screen looks has nothing to do with it. Format the human-facing view however you like, but do it as a separate, later step, because no amount of work at that end reaches back to change what the model already took in.

Reduction is a coupling, and coupling has a cost

Trimming is the right default, but it is not free of tradeoffs, and treating it as an unconditional good leads to a different failure. When you reduce a tool’s output to a fixed set of fields at the adapter, you are asserting that you know what every downstream step will need. That assertion couples the adapter to the agent’s future behavior, and coupling can be wrong.

The concrete failure mode is a later step that needs a field an earlier trim discarded. The agent reaches for information that a general response would have carried, finds it gone, and either fails or has to re-fetch. Aggressive, permanent trimming trades bloat for brittleness, and on tasks where you genuinely cannot predict which fields a later step will want, hard filtering at the adapter can starve the workflow of something it needed. This is the real boundary on the principle, and it is why the reduction problem has more than one shape. Some situations call for dropping fields outright. Others call for keeping the full payload reachable while keeping it out of the model’s immediate context, so nothing is lost but nothing irrelevant is in the way. Choosing between those approaches, and knowing when a workflow is short or predictable enough that the whole question does not matter, is its own design decision.

What does not vary is the diagnosis. The gap between what a tool returns and what an agent needs is a first-class problem in context design, not an implementation detail to sort out later. It degrades accuracy through misdirected attention, it compounds down the length of a tool chain, it charges you again on every downstream request, and it hides from the most natural place you would look for it. Recognizing that the reduction has to happen at the tool boundary, upstream of the model, before the output ever becomes part of the context the model reasons over, is the move that turns a slow, silent decline into a solved problem. The specific strategy for reducing it is a choice. Reducing it, wherever the output is large or noisy enough to hurt, is not.