When an extraction system attaches one confidence number to a record, it is reporting an average, and an average exists to hide variance. A record can post a strong overall score while one field inside it sits far below the line, and the aggregate erases exactly the fact anyone downstream needs: which value not to trust. The score worth acting on…
Instructing a model to work through a problem in the open, laying out intermediate steps before it commits to an answer, is one of the highest-leverage prompt changes available and one of the most casually overapplied. The leverage is real: on problems that decompose into steps, where an early mistake corrupts everything after it, making the model generate those steps first…
Writing a scratchpad is the easy part. A store that carries an agent’s state across sessions, whether that is a file in a coding agent’s workspace or a record in the datastore behind a deployed system, can be stood up in an afternoon, and on day one it does exactly what it promises. The difficulty is not authorship. It is keeping…
Finding something in a large system is a messy activity, and the mess is the problem. When an agent goes looking for the one file that handles a particular flow, or the one paragraph in a long reference that answers a question, it does not arrive there in a straight line. It searches, reads candidates, follows a lead that goes nowhere, backs out, and…
Least privilege is old advice. Scope every component to the access its own work demands and withhold the rest, so that a compromise anywhere stays confined to what that one component could already reach. The principle has governed the design of operating systems, service accounts, and API credentials for decades, and every senior engineer already believes it. With autonomous systems the principle…
Putting a human in the loop is the standard answer for the moments when an autonomous system should not act alone. It is also the answer that is easiest to get wrong, because the mechanism everyone talks about, the review step itself, is not where the design lives. The review is downstream. What actually determines whether human oversight protects anything is…
A measurement loop looks like rigor. You run the current version of a prompt against a set of cases, score each output, and compare the aggregate to the last version. The number that comes out feels objective in a way a reviewer’s impression never could. That feeling is mostly borrowed. An evaluation does not manufacture its own trustworthiness. It inherits it from…
A few-shot prompt is often treated as a writing task: draft a couple of examples that look right, paste them in, move on. That framing hides what is actually happening. A set of examples is a miniature training set, curated by hand and injected fresh on every call, and the model generalizes from it the same way it generalizes from anything it…
An agent has no memory of its own work. Each session opens with a blank context window, and everything the previous session learned, the approaches it tried and abandoned, the naming it settled on, the reasons behind the choices already made, is gone the moment that session ends. This is not a defect to be patched. It is the shape of how these…
A long-running agent will be interrupted. This is not an edge case to defend against at the margins; it is the ordinary weather of any task that runs long enough to matter. The process is killed by a deploy. A rate limit stalls a tool call past its timeout. A downstream service returns an error the agent cannot get past. A…