Every discussion of context as a scarce resource eventually arrives at what to do about the material already in the window: what to trim, what to summarize, what to replace with a reference. That is the downstream half of the problem. The upstream half is deciding what was allowed in to begin with, and it is the half that determines how…
Sooner or later a component in your tool layer needs judgment. A document tool fetches a fifty-page contract and has to hand back something that fits in the caller’s context budget. A retrieval tool pulls a hundred rows and has to decide which eight are relevant to the question that was asked. A ticketing integration receives a free-text…
Every failure in a running system is handled somewhere. It is swallowed by a component, absorbed by a retry, folded into a default value, or allowed to travel upward until something decides what it means. There is no such thing as an unhandled failure in production. There is only a failure whose handling location nobody chose. In most systems that location…
Confidence scores tell you which outputs you should not trust. Routing decides where those outputs go. The step that feels like the end of the design is the beginning of a second system, and most teams do not notice they have built one. The moment a pipeline can say “hold this one for a person,” it has acquired a dependency on…
By the time a model sees a prompt, it is a single string. Everything that went into it, the directives you wrote months ago, the records your retrieval layer pulled a few milliseconds ago, the text a user typed, the demonstrations you curated, the specification of the output you expect back, has been flattened into one undifferentiated run of characters. But those parts are…
A transcript is a record of what happened. The conversation history you hand a model is not a record. It is the evidence the model reasons from when it decides what to do next, and those are different artifacts with different requirements. Collapsing them into one object produces a specific, under-diagnosed class of failure: systems in which the accumulated past quietly…
An agent that escalates has already done the hard part. It reached the edge of its competence or its authority, recognized the edge, and stopped. That recognition is what every oversight mechanism in the system was built to buy: the thresholds, the retry ceilings, the policy checks, the confidence calibration. All of it exists to produce one moment where the system knows it should…
Every system that puts model output in front of a human reviewer is running a sampling policy, whether or not anyone designed one. Review capacity is finite and output volume is not, so some rule decides which fraction of the output a person actually looks at. The default rule, and the one that feels most defensible because it feels most…
Every instruction you give a model about the shape of its output is a request. You describe the artifact you want, and the model, weighing your description against everything it has learned about how a helpful reply begins, produces something that usually matches. Usually is the operative word, and for a system whose next step is a parser rather than a…
Inside a single engagement, memory is nearly free. Everything said, decided, and returned is already sitting in the context the model reads, carried forward by a default append that costs no one any thought. Retention is the cheap default and forgetting is the thing that costs you, which is why the entire discipline of managing a long conversation is about what you…