A Citation the Model Wrote Is Not a Record of Where the Text Came From

A synthesis system that reads twenty documents and returns a page of prose has performed a lossy, irreversible transformation, and the loss is not in the words. It is in the correspondence. Somewhere in the generation step, a specific passage in a specific document became a specific sentence in the output, and by the time a reader sees that sentence, the relation is gone. Nothing in the text carries it. The fluency of the result actively conceals the loss. Prose that rests on twenty documents and prose that merely sounds like it does are, on the page, the same prose.

Provenance is the discipline of not losing that correspondence in the first place. It is routinely mistaken for a presentation concern, a matter of putting markers next to sentences so the output looks sourced. That framing gets the architecture backwards and produces the most dangerous artifact in the whole space: output that carries the visual signature of being traceable without the property. The markers are not the mechanism. They are a rendering of a mechanism that either exists upstream or does not exist at all, and when it does not, they are simply more generated text, produced by the same process, subject to the same failure modes, and trusted more.

A model asserting a source is making a prediction, not filing a record

The decisive question about any citation is what produced it. If the model emitted the marker as part of generating the sentence, then the marker is a prediction about which source is plausible given the sentence, drawn from the same distribution that produced the sentence. It was not read out of a ledger. Nothing consulted the retrieval step to check. A confident, well-formatted, entirely fictitious attribution is not a malfunction. It is what the mechanism produces on an input where the plausible answer and the true answer diverged.

This is why “the model cites its sources” and “the output is traceable” are unrelated claims that get spoken as synonyms. Traceability is a property of the system’s bookkeeping. The retrieval layer knows precisely which passages it put in front of the model, with what identifiers, in what order. That knowledge is authoritative and free, and it is usually thrown away the moment the prompt is assembled. Reconstructing it afterward from the model’s own testimony is a strange thing to do when the ground truth was in hand a moment earlier.

The stronger design treats attribution as a separate determination made against the retrieved set rather than a byproduct of generation. The candidate passages are known. The output claim is known. Whether that passage supports that claim is a question with an answer, and it can be settled by a check that has no stake in the prose reading well: an entailment judgment, a scoped comparison, a second pass whose only job is to decide support and which is permitted to return “none of these.” That check may itself use a model. The point is not to purge models from the loop. It is that the component asserting the link is not the component that wanted the sentence to sound authoritative, and it can fail loudly by returning nothing, which the generating pass structurally cannot do.

The economics of that separation are the part worth internalizing. Attribution asserted during generation costs nothing and is worth nothing. Attribution determined by an independent check costs a second pass over every claim and is worth what it costs. There is no configuration in between, no prompt that converts the first into the second, and a great deal of engineering effort goes into looking for one.

The unit of the pointer is the unit of the repair

Granularity gets argued as a question of reader convenience, markers beside sentences against a list at the end of a section. It is really a question about what the system can do when a claim turns out to be wrong, and the answer is determined long before anyone is reading.

Attach sources to a paragraph and you have recorded that some subset of these documents informed some subset of these sentences. When a reader disputes the third sentence, that record cannot tell you which document to open. It has narrowed the search from twenty documents to four, which is a real reduction and a nearly useless one, because the expensive part was never finding the document. It was establishing whether the specific assertion is supported, and that requires knowing which assertion was meant to come from where. Paragraph-level provenance answers a question nobody has.

Claim-level provenance makes the individual assertion the addressable thing, and that changes what correction costs. A wrong claim with a pointer is a local defect: follow the pointer, see that the passage says something narrower, fix the sentence, leave the rest of the document alone. A wrong claim without one is a defect in the document, because you cannot establish that the surrounding sentences do not share its cause without checking them, and the honest response is to regenerate and revalidate the whole thing. One is a repair. The other is a recall.

That property compounds in the direction people underestimate. Synthesis is layered: outputs feed summaries, summaries feed decisions, and an early claim becomes a premise nothing downstream reexamines. Without claim-level addressing there is no way to ask which conclusions rest on the sentence that turned out to be wrong. The question has no shape. You are left rerunning everything or accepting that some conclusions are load-bearing on a fact you have retracted, and in practice the second thing happens, silently, because the first is expensive and nobody can say which conclusions are at risk.

The schema is a set of claim types, not a claim and a list of pointers

The naive mapping is a table of claims and their source identifiers, and it holds up until the first claim that is not a restatement of any single passage. Those arrive immediately, because a synthesis that only restates passages is not a synthesis.

A claim that aggregates across sources has no supporting passage. No document says the figure rose in four of the five regions; five documents each report a region, and the claim is arithmetic performed on top of them. A claim of absence is worse: nothing in the corpus mentions the exclusion, and the evidence for the assertion is a property of the whole retrieved set rather than a location within it. A claim that follows by inference from two passages that never appear together is supported by neither one alone. And some claims are simply the model’s own connective reasoning, the tissue that makes the output a document rather than a list.

Force a schema that demands a pointer per claim onto that reality and it will be satisfied. The system will attach the closest available passage to a claim that passage does not support, and that outcome is worse than an unsourced claim by a wide margin. An unsourced claim is visibly a claim, and a reader treats it as one. A falsely sourced claim has been laundered. It is now carrying a credential, it will survive review by anyone who checks that a pointer exists rather than that it lands, and it is indistinguishable from correct work at exactly the moment of the check.

So the schema has to admit what the claim actually is. Directly supported, with the passage. Derived, with the inputs it was computed from and the operation performed. Inferred, with the passages it was inferred from and an explicit acknowledgment that the step is the system’s. Asserted by the model with nothing behind it, which is a legitimate category that must be representable, because if it is not, its members do not disappear. They get filed under one of the other three.

Support has degrees, and the interesting failure is overreading a real source

Once the mapping distinguishes claim types, the residual work is in the relation itself, and it is not binary. Between “this passage states this claim” and “this passage has nothing to do with this claim” is a wide band where most real problems live.

The passage says a trial found an association; the claim says the treatment causes the outcome. The passage reports one quarter; the claim describes a trend. The passage attributes a position to a named party; the claim states the position as fact. The passage hedges; the claim does not. In every case the pointer is honest. The document is real, it was retrieved, it is on topic, and following the pointer lands a reader on text that is recognizably about the claim. The claim is still wrong, and it is wrong in the direction of confidence, because the transformation that generated it was optimizing for prose that reads well and hedges read badly.

Invented sources get the attention because they are lurid, and they are also the easy case: an identifier either resolves or it does not, and a system that never asks the model to produce identifiers cannot invent them. Overreading is the hard case. It survives every structural check. The identifier resolves, the passage is relevant, the format is correct, and a reviewer skimming for whether claims have sources will pass it. Catching it requires actually comparing the strength of the assertion to the strength of the evidence, which means the support determination has to record a relation and not just a link. Stated. Implied. Consistent with but not established by. Contradicted. A mapping that only records that a link exists cannot express the difference between a claim and its source’s actual position, which is precisely the difference that matters.

An identifier that does not survive the document is not a pointer

Provenance rests on identifiers, and identifiers are a harder problem than they look, because the thing they name moves.

Point at a document and a section offset, and the pointer degrades the moment the document is revised. It will not break loudly. It will keep resolving, to different text, and a reviewer following it a month later reads a passage that does not support the claim and concludes the system fabricated an attribution that was accurate when it was made. Point at a chunk index from the retrieval layer and the pointer is a coordinate in an ephemeral coordinate system, correct until the corpus is reindexed with a different chunking strategy, after which every stored pointer in the history is confidently wrong. That reindex will happen. It is a routine operational change, made by people improving retrieval quality, who have no reason to know they are invalidating an audit trail.

The property a pointer needs is that it names the version of the text that was actually read, not the current occupant of a location. That means content addressing, or a snapshot of the cited span stored alongside the mapping, or a corpus with real versioning and pointers that carry a version. Which of those you pick is a genuine engineering tradeoff. That you need one of them is not.

And the commitment runs further than most designs anticipate, because a pointer to a version implies retaining that version. Provenance imposes a retention policy. It sits uncomfortably beside any obligation to delete source material, since a mapping whose targets have been deleted is a trail that ends in nothing, and a mapping that preserves quoted spans of deleted material has moved the material rather than removed it. There is no clean resolution to offer here, only the observation that this is a decision, it belongs to whoever is designing the trail, and it is far cheaper to make it deliberately than to discover it during the first request to delete something.

The trail is more valuable read backwards

Claim-to-source is how provenance is described and how it is used least. The mapping is a bipartite graph, and the direction that earns its keep is the one that is rarely built: source to claim.

A source is retracted, corrected, superseded, or found to have been misread. The operational question at that moment has nothing to do with any particular output. It is: what have we said that rests on this? Which conclusions, in which documents, delivered to which consumers, depend on a passage that no longer holds. A per-document claim list cannot answer it. Answering it means an index over the mapping keyed by source, spanning every output the system has produced, and that index is a straightforward thing to maintain from the moment the mapping exists and an archaeological project to reconstruct afterward.

The systems that have this can do impact analysis, and the difference in what they can promise is stark. They can enumerate exactly what is affected by a bad source and correct precisely those claims. The systems without it face a choice between reprocessing their entire corpus of outputs and doing nothing, and they choose doing nothing, because the first option is not affordable and the second is not visible. That is the actual payoff of provenance as an architecture rather than a display convention, and it is available only if the mapping is a durable store rather than a decoration on a rendered document.

Provenance constrains what the generator is allowed to produce

None of this can be retrofitted onto free-form generation, and the attempt is the most common failure in the space. A pipeline that produces a page of prose and then asks a second pass to attribute it is trying to reconstruct a correspondence that the first pass destroyed. The second pass will do something. It will find plausible-looking support for most sentences, because for most sentences a plausible-looking passage exists among the twenty documents. That is not recovery of a lost mapping. It is a fresh, independent fabrication with a strong prior toward looking right.

Provenance has to be a constraint on generation. The output must be produced as discrete, addressable claims with their support determined as they are made, and only then rendered into whatever prose the reader sees. The prose becomes a view over a structured artifact rather than the artifact itself. This is a real architectural commitment and it has real costs, which deserve to be stated plainly rather than discovered.

Structured claim-by-claim generation produces flatter, more clipped prose than free synthesis, because the sentence that gracefully weaves three sources is exactly the sentence that resists having one pointer. The system will hedge more, because a model required to ground each claim will retreat toward whatever it can ground, and some of the synthesis you wanted was in the connective steps it now declines to take. Every claim is a support determination, so cost and latency rise roughly in proportion to how much the output says. And the mapping is a durable artifact with its own storage, schema, versioning, and lifecycle, which is a second store to operate alongside the first.

Those costs buy one thing: the ability to establish, later, whether any particular sentence is true. Where that ability is not worth a second store, do not build one. A summary read once and discarded, a draft a person will rewrite, an answer whose sources are the answer anyway because the consumer wanted the documents and not the prose about them. The mistake is not declining provenance. It is declining it while still rendering markers that suggest you have it.

The pointer is a promise that someone could check, and the promise is the liability

The value of a claim-to-source mapping is not realized when it is built. It is realized when someone follows a pointer and finds that the passage says what the claim says, or finds that it does not. Everything upstream of that moment is preparation, and a mapping nobody ever follows has changed nothing about whether the output is correct. It has only changed how correct it appears.

Which is the reason to be uneasy about provenance as a feature rather than a discipline. A marker beside a sentence is an assertion that the sentence can be checked, and readers respond to that assertion the way they respond to any credential: they extend trust and they check less. If the assertion is backed by an independent support determination against a stable, retained identifier, that trust is well spent and the system has earned a real reduction in the reader’s burden. If it is backed by the model’s own guess about which source sounds right, the system has manufactured the trust without the property, and the sourced output is now more dangerous than the unsourced version was, because the unsourced version at least prompted the reader to wonder.

Treat every claim as a first-class object with a type, a support relation, and a pointer that names the version of the text it was read from. Determine the support with something other than the process that wrote the sentence. Index the whole thing by source, so the day a source fails you can say what it took down with it. Then, and only then, render the markers. The output is auditable when the correspondence between text and evidence is maintained by the system as a matter of record, and no amount of formatting substitutes for it. Every notation that goes into a document without that mechanism behind it is not a citation. It is a costume.