Conversation History Is State You Own, Not a Transcript You Append To

A transcript is a record of what happened. The conversation history you hand a model is not a record. It is the evidence the model reasons from when it decides what to do next, and those are different artifacts with different requirements. Collapsing them into one object produces a specific, under-diagnosed class of failure: systems in which the accumulated past quietly argues against the current instruction, and wins often enough to matter.

The transcript metaphor is what makes the collapse feel natural, and its naturalness is why so few teams ever question it. What the model sees on any given request is assembled by the application, out of material the application chose to keep. That assembly is the input you have the most authority over, and in most systems it is the input nobody owns.

The history is a variable, and every turn resubmits it

Nothing that enters a conversation is written once. It is written once and then read again on every subsequent request, at full length, until something removes it. The fortieth turn of a session carries the first thirty-nine with it. This is well understood as an economic fact, and the economics are the least interesting consequence.

The interesting consequence is that a variable is mutable. You are not obligated to send what happened. You are obligated to send what helps. The message list handed to the model on turn forty does not have to be an accurate chronology of turns one through thirty-nine, and in a long-running system it almost certainly should not be. Very few systems ever exercise that freedom, because the framework hands them an array and an append operation, and an append operation is a strong suggestion about how a thing is meant to be used.

Everything in the window reads as evidence

A message list has no way to encode status. There is no slot for saying that a statement held once and stopped holding, that an approach was tried and abandoned, that a request was made and then taken back. Those distinctions exist in the mind of the person who watched the conversation unfold. They do not exist in the request.

Position and recency carry some weight, and a model will often infer supersession correctly when the conversation makes it obvious. But that inference runs over the same channel as the content itself. It is a soft reading of the text, competing with everything else in the text. Nothing in the request marks a message as expired, and a constraint that still governs looks structurally identical to one that was withdrawn twenty turns ago. Everything is admitted on equal terms, and the model’s job is to reconcile all of it into one coherent next action.

Most of the failures that follow are the same failure. The system asks a model to act on a body of evidence that contains its own contradictions, its own dead ends, and its own mistakes, and then treats the resulting inconsistency as a shortcoming of the model.

An overridden instruction is still in the request

Consider what happens to a constraint over the life of a session. It is established early, in the framing, where it carries the most structural weight. Something changes: an operator relaxes it, a policy lookup returns a different value, a user amends their own request. A new instruction appears later in the history. The old one does not go anywhere.

The request now carries two versions of the same rule and no structural signal about which one governs. What follows is worse than a clean failure. The model resolves the conflict correctly most of the time, and reverts under pressure: when the superseding instruction sits far away, when the original is stated more forcefully, or repeated, or reinforced by the several turns that were executed under it. Intermittent failure does not present as a broken system. It presents as an unreliable model.

The response to an unreliable model is usually to prompt harder, which in practice means restating the instruction. That adds another copy of the constraint to the history. Now the request contains an old constraint, a new constraint, and an emphatic restatement of the new one, and the ratio of instruction to task has shifted again. The remedy is feeding the disease.

The actual fix is to stop layering. When a constraint changes, the new value of that constraint should replace the old one in what you send. A constraint has a current value and the request should contain exactly that, once.

A correction does not remove the thing it corrects

When a turn produces a wrong artifact, a wrong plan, a malformed record, an extraction that invented a field, the reflex is to append a correction. The wrong artifact stays.

Look at what the model is then asked to weigh. On one side, a fully specified, concrete, often lengthy piece of output, sitting in the position of an accepted result. On the other, a short instruction to disregard it. The specificity gradient runs backwards. The mistake is more detailed than its retraction, and detail is what a model has to work with. Worse, the mistake is now precedent. It has established a format, a level of granularity, a set of field names, an approach. Later turns will pattern-match against it even when they have been told not to, because it is the most concrete example of the target artifact in the window.

Appending is not retraction. If a mistake should not influence what comes next, it has to leave. The correction turn is a repair to the conversation’s surface, not to its state.

Abandoned work keeps charging rent

A meaningful share of any agentic run is exploration that leads nowhere. A retrieval that surfaced the wrong records. A plan that was reconsidered after two steps. A delegated agent that came back with nothing usable. This is not waste; it is how the work gets done. The waste is what happens afterward.

When a branch is abandoned, its conclusion has real value. That source does not contain the record. That approach fails for this class of input. That conclusion is small, durable, and worth carrying for the rest of the session. The trace that produced it, the tool calls, the intermediate reasoning, the partial outputs, has almost no value once the conclusion is known. Yet the default behavior keeps the trace, in full, forever, and lets the conclusion dissolve into it. The one part worth carrying is the part that gets lost, and everything else is carried faithfully to the end of the run.

Editing the history is available, and almost nobody does it

The prescription follows from the diagnosis. Treat the message list as a rendering of state, not as the state.

Once it is a rendering, a set of operations the append model never offered comes into reach. A changed constraint is updated in place rather than layered. A failed attempt is deleted and replaced by what it established. A resolved sub-thread collapses into its conclusion. An artifact appears at its current version rather than trailing every draft that preceded it. The invariant is that what you send must be true, current, and sufficient, and none of those three requires it to be in the order events actually occurred.

Behind the rendering sits a working set the application maintains and updates: the task as currently understood, the constraints at their current values, the decisions and the reasons for them, the artifacts at their latest version, and what remains open. Rendering that into messages costs a step at request-assembly time and buys a model input that is a considered account of where the work stands rather than the raw sequence of everything that has happened so far.

The record you owe and the input you send are different artifacts

The immediate objection to any of this is auditability. You cannot go around deleting things. Regulated workflows need to show what happened. Incident review needs the real sequence. Evaluation needs the actual inputs. All of that is correct, and none of it is an argument for accumulation. It is an argument against making one object serve two purposes.

Keep the log as a log: complete, ordered, immutable, retained by policy. Every message, every tool call, every result, every curation event, including what was removed and why. Send the curated view. The moment those two are the same object, one of them is wrong. Either the log is missing things that really happened, because you were willing to edit it, or the model is reasoning over material it has no business reasoning over, because you were not.

Separating them is also what makes aggressive curation safe. The usual fear about pruning is that you will throw away something that turns out to matter, and that fear is well founded when the input is the only copy. When the log is intact, a bad curation decision is a bug you can diagnose and replay rather than an irreversible loss. The log is not a compliance tax on this design. It is the thing that licenses it.

Prefix reuse makes appending cheap and rewriting expensive

There is a real cost on the other side. Reuse of an unchanged prefix is what makes long conversations economically viable. A request that only adds to the end of the history can reuse the work already done on everything before it. A request that changes something early invalidates that reuse from the point of the edit onward. Append is cheap by construction, and the edits with the highest value, superseding a constraint, removing a mistake, rewriting the framing, are exactly the ones that land early and break the most.

This is a genuine tension, and it resolves into a scheduling decision rather than a reason to give up. Do not curate on every turn; that pays the cost constantly and captures little. Curate at boundaries, when a sub-task completes, when a branch is abandoned, when a constraint changes, when a large result has been fully consumed. At those moments, accept the cost of a rebuild in exchange for a prefix that is smaller and more stable than the one it replaces, and that then stays stable across the stretch of turns that follows. A rebuild that meaningfully shrinks the history pays for itself over the turns that would otherwise have carried the excess.

There are cases where plain accumulation is correct. Conversations short enough to end before any of this matters do not need the machinery. Conversations where the history is itself the product, displayed to a person who expects to see what was actually said, have a reason to keep it faithful. And a system with no reliable basis for deciding what to drop is better off accumulating than guessing, at least until it has one. Curation is a claim about what is no longer needed, and a system that cannot support that claim should not be making it.

Continuation needs a reason

Sessions continue because nothing stopped them. That is not a design.

The question to put to a running session, periodically and on purpose, is whether what it carries still earns its place in the next request. That is an evaluable question, not a feeling. Are the constraints in the window the ones actually in force. Is the artifact being referenced the current one. Would a rebuild drop anything the next step genuinely needs. When those answers hold, continue. When they do not, the history has stopped being an asset and become an argument the next instruction has to win.

A full reset is rarely the right instrument, because it discards work that was expensive to produce. A partial rebuild almost always is, and the test for what survives one is simple. Keep whatever the next step will be held to. Drop the process that arrived at it. Deliberation is how the conclusions were reached; it is not one of them.

In a served system this is an internal operation, not a user-visible event. Same session, same user, same task, new working set. Nobody should have to start a new conversation because the system could not manage its own state, and nobody should be asked to repeat themselves so that it can. The rebuild is the application’s job precisely because the history was the application’s to begin with.

What survives a deletion should be the lesson, not the trace

Curation has a failure mode of its own, and it is the one that discredits the whole practice when it goes wrong.

Naive pruning drops the two things that were most expensive to acquire: the reason behind a decision, and the negative result. A constraint that arrives without its rationale has no defense, so the model re-opens it, or quietly reverses it, or honors the letter of it in a way that defeats its purpose. A failed attempt that is deleted cleanly gets attempted again, enthusiastically, by a system that has no idea it ever failed. Losing both is how a curated session starts to loop.

So the rule is not to delete what failed. It is to replace what failed with what it taught. Curation that only subtracts is compression, and compression is lossy in exactly the places that cost the most. Substitution, trading a long trace for the short finding it produced, is what makes a long-running session cheaper and sharper at once. Subtraction alone makes it cheaper and dumber.

The history is the one input you fully control

Almost nothing else about a request belongs to you. The model behaves as it behaves. Users phrase things how they phrase them. Tools return what they return. The history is different. Your application writes nearly all of it, since tool results, retrieved documents, and reports from delegated agents are far bulkier than anything a person types, and your application reassembles the whole thing on every call. How fast a conversation grows is therefore a fact about your design rather than about your users, which puts the lever on your side of the boundary. Letting the history become a passive byproduct of the interaction is declining to pull it.

Pulling it asks nothing new of the model. The message list becomes a rendering of state rather than the state itself. Corrections replace what they correct instead of stacking on top of it. Constraints have current values rather than histories. The audit log and the model’s input are separate artifacts with separate rules. Continuation happens because someone chose it, not because no one intervened. A system built on those terms does not accumulate its way into incoherence, because it was never accumulating. It was deciding, every turn, what the model deserves to see.