The Persistent Fact Store Is a Subsystem, Not a Feature

Once you accept that a summarizer will drop the exact values a long session depends on, the fix is easy to describe and easy to half-build. Pull the facts out of each turn before compression runs, write them into a durable region the summarizer never touches, and the values survive while the narrative is free to shrink. Stated that way, a persistent fact store sounds like a single feature. It is not. It is a small subsystem with at least five distinct decisions inside it, each with its own failure mode, and the decisions are coupled rather than independent. A store can be built correctly in four of them and still fail completely because the fifth was skipped. That is the trap worth understanding before writing any of it: the extract-first idea is trivially cheap, and the store that implements it well is not.

The five decisions are what to extract, how to update the store when a fact changes, where to place the store in the request, how to keep it from growing without bound, and how to make the model actually read it. None of these is optional, and none can be tuned in isolation without pushing a cost onto another. What follows is each decision in turn, and then the reason they have to be treated as one system.

The extraction prompt decides what the store is worth

Everything downstream inherits the quality of the extraction step, so it is where a store most often quietly fails. The failure is rarely dramatic. When the instruction is vague, the output is vague to match. Ask only for whatever the model judges significant and it applies a narrative sense of significance, one tuned to what reads as memorable rather than to what the application will later key on. It keeps the gist and lets the identifier go, the exact loss the store exists to prevent. Handing the judgment of what matters back to the same kind of process that discards facts under compression defeats the purpose before a single value is written.

The correction is to take the judgment away from the model and encode it in the prompt. Name the categories outright. The classes a compressor most reliably throws away are the concrete ones: named entities, the identifiers that point at them, the numeric limits that constrain them, and the decisions already committed to. A prompt that asks for exactly those, instead of for whatever seems important, yields output whose shape is fixed in advance and whose coverage can be reasoned about. The model is no longer choosing what to keep; it is populating slots defined for it.

Two properties of the output matter as much as the categories. The first is that it should be typed and structured rather than loose prose, because the whole point of the store is to hold values that get validated and consumed by code, and a structured record is something a program can check where a paragraph can only be reread. The second is subtler and worth more than it looks: for any category the turn said nothing about, the extractor should emit an explicit null rather than drop the field. A missing field is ambiguous. It might mean the category was examined and found empty, or it might mean the extractor never looked. An explicit null settles that, recording that the step ran this turn and came back empty, which is a different and more trustworthy state than silence. Across a long session, the gap between “checked, nothing there” and “no idea” is the gap between a store you can rely on and one you can only hope about.

Whether a fact’s history matters is decided fact by fact

Once facts are being extracted, the store needs a rule for what to do when something it already holds reappears with a different value. Two base policies exist, and they serve opposite needs. Append-only writes every stated value as a fresh entry and overwrites nothing, keeping the complete record of how a fact moved across the session at the cost of a store that swells every turn. Replace-on-conflict retains only the newest value for each fact, staying compact at the cost of discarding every earlier value for good. Neither wins in the abstract. The right choice follows from one property of the fact itself: is its past worth keeping, or is only its present state ever read?

For some facts the history is the whole point. When a user resets a spending limit midway through a session, a later step may need to know not just the current number but that it changed at all, and sometimes why. Collapse the fact to its latest value and that information is gone. For other facts the history is pure noise. When a user fixes a misstated field name, only the fixed version has any use, and dragging the earlier mistake forward is a standing liability that tempts the model to surface it again. This is why a mature store rarely runs one policy end to end. It partitions: some facts are declared fixed and only ever appended, others are declared revisable and carry only their latest value. Which side a fact falls on is not chosen at runtime. It is fixed by the extraction schema, so the schema and the update rule are one decision viewed from two sides. You cannot pin down how the store updates without first pinning down what kinds of facts it holds, and that dependency is the first hint that these layers are not really separable.

Where the block sits determines whether the model trusts it

A fact store is a region of the prompt, and where that region sits changes how authoritatively the model reads what is in it. Prepend it to the user turn and it stays in view, but it now sits beside the actual question and draws attention away from it, and the dialogue ends up carrying structural data that was never really part of the conversation. Drop it directly ahead of the query and it becomes hard to miss, at the price of a longer, noisier turn and a blurred boundary between what is context and what is being asked. Set it apart instead in the structural, system portion of the prompt, away from the conversational turns entirely, and it reads as standing context rather than as something the user just said. For most architectures that separation is what leads the model to treat the store as a source of truth instead of as one more conversational contribution to weigh.

The reason this is a real decision and not a formatting preference is that the model’s treatment of information is sensitive to where that information appears. Facts buried in the flow of dialogue read as things said in passing. Facts set apart in a dedicated structural region read as the ground the response is supposed to stand on. The store’s whole value is that it is authoritative, and placement is one of the levers that establishes authority. Get it wrong and the store can be present, accurate, and still discounted because the model reads it as conversational chatter rather than as the record it is meant to be.

Conflicts have to be detected, not silently resolved

The update policy says what to do when a fact changes, but it presupposes that the change was noticed. In any session where the same piece of information is stated more than once, the values will sometimes disagree, and the dangerous case is not the disagreement itself but a store that resolves it without recording that it happened. If a conflict goes undetected, one of two bad outcomes follows. Either the original value stays and the correction is silently lost, or the new value overwrites the old one with no record that anything changed. Both hide a data-quality event that will resurface later as an application bug whose cause is invisible, because by the time the symptom appears the evidence of the overwrite is gone.

A store worth building catches the conflict before it writes. It checks the incoming value against what it already holds, and when the two disagree it records both, applies its update policy on purpose rather than by accident, and raises the question to a human when the fact in conflict is one whose value has real downstream weight, a spending cap, an access threshold, a bound that some later computation depends on. The escalation matters because conflicts are not all equal. A fixed typo can be resolved silently and safely. A changed limit should not be, because if the change was an error or a misheard turn, absorbing it without a flag turns a recoverable moment into a permanent wrong value that every later answer is now built on. A silent overwrite buys quiet now and pays for it later with a defect whose origin has already been erased, and the store’s job is to decline that bargain on the facts that carry weight.

Extraction can be wrong, so the store needs a check

Extraction is itself a model operation, so it can misread a stated value, bind a quantity to the wrong subject, or fail to register something the turn made explicit. A store that trusts its extractor without reservation absorbs every such error and then grounds each later response on it, which is worse than the original loss, since a false value asserted plainly gives fewer signs of trouble than an obvious gap does. So the store needs a validation step between extraction and the moment a value is allowed to count as authoritative.

Two checks do most of the work. The first is type and bound enforcement at write time: a field declared numeric refuses a string, a turn index below zero is rejected, a value sitting outside any plausible range never reaches storage. Mechanical distortions die here, cheaply. The second reaches the errors that pass every type check, by running extraction in reverse. Take what the store now holds, regenerate the facts it implies, and set them against the original turn; where the regenerated version and the turn disagree, the extractor bent something on the way in, and the disagreement surfaces it while it is still a data-quality question and not yet buried in application logic. Both checks add a little latency, and they earn it by targeting the single hardest failure to diagnose after the fact: a value that was extracted and written, looks entirely plausible, and is simply not what the user said, quietly shaping every response that follows with no error anywhere to trace. The store is worth trusting not because extraction never errs but because a written value can always be rechecked against the turn that produced it, and these two checks are what make that recheck routine rather than theoretical.

Size is a budget you manage on purpose

A store that only ever grows will, in a long enough session, consume the very context budget it was built to protect, at which point it has recreated the pressure that summarization was relieving. Bounding it is not an optimization to add later. It is a load-bearing part of the design, and it interacts directly with the update policy: an append-only store grows faster and needs harder bounding than one that replaces in place, so the choice made earlier about history sets the difficulty of the choice here.

Four strategies address the growth, and they compose rather than compete. The bluntest is a fixed ceiling: past a set token allowance the store refuses new facts until something older is evicted, crude but absolute on the upper bound. A second retires facts by disuse, stamping each with the last turn it was read and dropping the ones that have sat unread across some window, on the bet that what nothing has needed lately is the safest thing to lose. A third folds repeated references to a single entity into one record, clearing the duplication that builds up as a session keeps returning to the same subjects. A fourth ranks facts, walling off the ones that can never go, the task’s defining goal, who the user is, the constraints the whole job rests on, from the disposable remainder. Most durable systems run two or three of these at once rather than leaning on any single mechanism, because each addresses a different way a store swells and none covers them all. The discipline that keeps size in hand is the same discipline that keeps extraction selective: the store guards the facts that matter and releases the ones that do not, and a store that hoards indiscriminately has preserved nothing worth its cost, it has only grown back into the problem it was built to solve.

A store the model is never told to read is dead weight

The most common way a fact store fails is also the most avoidable, and it survives even careful implementations, which is why it is worth stating plainly. A team writes a precise extraction prompt, chooses a sound update policy, places the block well, and bounds its growth, and then never adds the instruction telling the model to consult it. The store is present. Its facts are correct. And the model still answers from what it already knows and from the running dialogue, because nothing in the request told it that the block is authoritative or that its contents should ground the response. Every other layer worked, and the store is still dead weight, spending tokens to hold facts that never reach an answer.

The fix is not clever and it is not optional. Every request that carries the store has to carry an explicit instruction pointing at it and saying what to do with it: that a designated section holds established facts, and that the response should be grounded in them. Without that instruction, the model has no reason to privilege the store over anything else in the context, and a store the model does not privilege is decoration. This failure hides so well precisely because the store looks complete. Nothing is missing, nothing errors, the facts are right there to anyone inspecting the prompt. The gap is not in the store at all. It is in the connection between the store and the generation step, and that connection is a sentence in the prompt that is easy to consider too obvious to write.

The layers are coupled, so the store is right or it is not

Reading these decisions as a checklist misses the thing that actually makes a fact store work. They are not five independent knobs. The extraction schema determines what facts exist to hold, which determines what the update policy can even distinguish. The update policy determines how fast the store grows, which sets the difficulty of bounding it. Placement determines whether the model treats the store as authoritative, which the consult instruction then depends on to mean anything. Change one and the pressure moves to another. This is why a store that is excellent in four decisions and absent in the fifth does not degrade gracefully to eighty percent of a working store. It fails, because the missing layer breaks the chain the other four were links in.

The upside of that coupling is the reason to build the store at all. Make each layer on purpose, specify extraction instead of delegating it, match the update policy to the nature of each fact, place the block where it will be trusted, govern its size actively, and tell the model in so many words to read it, and fact loss stops being an inevitable cost of long sessions and becomes a solved problem. The extract-first insight is what makes the store possible. The five decisions are what make it real. Neither suffices without the other, and the store is worth its cost only when both are handled with the seriousness of the subsystem it actually is.