Structure Can Be Guaranteed. Truth Has to Be Checked.

A language model can hand you an object that is perfectly shaped and entirely false. Every field is present, every type is right, the whole thing parses without complaint, and the value in the field your downstream logic depends on was invented. This is the failure mode that survives structured output. Enforcing a schema settles what shape the answer takes. It settles nothing about whether the answer is correct. And the defense against a well-formed fabrication is not a stronger prompt or a stricter schema. It is a loop: generate, check the result against something the model cannot argue with, feed the specific failure back, try again within a fixed budget, and escalate when the failure refuses to resolve. That loop is what stands between a pipeline that behaves on the inputs you tested and one that keeps behaving on the inputs you never anticipated.

The reason the loop matters is that the guarantees you can buy at generation time and the guarantees you actually need do not overlap as much as they appear to. Constrained decoding, where the schema is compiled into a grammar that the decoder cannot violate, removes structural error completely. A required key cannot go missing and a typed field cannot arrive as the wrong type, because no path through generation produces those results. That is a guarantee you keep for good, and it is worth having. But it operates entirely on form. It has no view into whether the extracted date is the date that appears in the document, whether the referenced identifier corresponds to anything real, or whether the summary summarizes the right thing. The model can satisfy every structural constraint and still populate the fields with values it simply made up. Structure is a solved problem the moment you engage constrained decoding. Correctness is the problem that remains, and it is the harder one.

The structural layer is the one you get for free

It helps to see validation as three distinct layers, because each catches a different class of failure and teams almost always build the first one and stop.

The first layer is structural conformance. The output has the declared fields, the declared types, the declared shape. This layer is fast, deterministic, and fully automatable, and it is exactly the layer that constrained decoding already guarantees when you use it. That is the point worth internalizing: if generation is constrained against the schema, structural validation is largely redundant, because the failures it looks for cannot occur. Investing your validation effort here is spending on a problem the generation mechanism already closed. The teams that validate structure and ship are not wrong that structure is validated. They are wrong that structure was ever the exposure.

The second layer is domain validity. The values are structurally fine and also obey the rules of the domain they describe. A timestamp resolves to a moment that could actually belong to this kind of record. A monetary amount stays inside the bounds the account it posts to could ever hold. A status code is one the workflow actually defines rather than one the model coined. A pair of fields that must agree, a start that precedes its end, a total that equals its parts, actually agree. Structural validation reaches none of this, because a value can be flawless as data and absurd as fact: a perfectly typed timestamp can name a moment this document could never carry, and the type check passes it straight through. This layer requires you to codify domain logic explicitly, in code, as rules the output has to pass. It is more work than a schema check and it is the layer that catches the errors a schema check is blind to.

The third layer is source fidelity. The values are not merely plausible and well-formed, they are actually supported by the evidence the extraction was run against. This is the layer where fabrication lives. A model asked to pull a figure from a document that does not contain that figure will frequently supply a figure anyway, drawn from the statistical shape of similar documents rather than from this one. The number is plausible, correctly typed, and inside the valid range, so it passes both prior layers untouched. The only thing wrong with it is that it is not true, and the only way to catch it is to check the value back against the source. This is the layer that separates a pipeline you can trust from one that merely looks structured.

Most extraction pipelines under-invest in the second and third layers precisely because the first is so easy and so visible. Schema validation produces a satisfying green light. It is also the light that tells you the least about whether the output is right.

The reprompt is where the loop earns its keep

Detecting a failure is only half of a validation loop. The other half is doing something useful with the failure, and the quality of that half is decided almost entirely by the reprompt.

A generic instruction to try again is close to worthless. Nothing in it tells the model where the first answer went wrong, so the second pass runs the same reasoning over the same inputs and lands in the same place, occasionally dressed in different words that pass for a fresh attempt while failing the exact check that caught the first. A retry that carries no information about the failure is not a correction, it is a re-roll.

An effective reprompt does three jobs at once. It states the exact failure, which check tripped and what a passing value would have to look like, so the model aims at a defined target instead of guessing at what displeased the checker. It puts the source material back in front of the model, so the next answer is drawn fresh from the evidence rather than patched out of the first answer’s memory, which is how a model quietly recycles its own fabrication into a revised one. And it confines the correction to the fields that failed. Regenerating the whole object to repair a single field is a common and expensive error: the fix can land while new mistakes appear in fields that were right to begin with, and a loop that rewrites everything each pass can swing from one failure to another without ever settling. Correcting only what broke holds each retry’s reach to the failing field and keeps the chance of fresh damage low.

The through-line is that the correction can only be as precise as the feedback that provokes it: name the fault exactly and the revision has somewhere to go, gesture vaguely and the revision only shuffles the surface. The reprompt is not a formality wrapped around the retry. It is the mechanism by which the retry has any chance of succeeding, and a loop with a weak reprompt is a loop that spends money to fail more slowly.

A retry budget, and the failures no retry will fix

A validation loop without a hard cap on attempts is a latent outage. A persistent failure becomes an unbounded sequence of retries that drains cost and blocks the pipeline behind it, and the failure that triggers it is often exactly the kind that will never resolve on its own. So the budget is not a tuning parameter to add later. It is part of the loop’s definition.

The cap should be small. A couple of attempts recover the overwhelming majority of genuinely transient failures, the ones where a single retry with a good reprompt lands the correction. If an output cannot pass in that many tries, further attempts rarely help, because the cause has shifted from a one-off stumble to something structural in the prompt or the task. Keep a record of each failed attempt and the check it missed, because that record is what later exposes the systematic weaknesses in your instructions, the failures that only look random until you notice the same prompt defect surfacing across many inputs.

This points at the most important judgment in the whole pattern: when a meaning-level miss recurs on attempt after attempt, the loop has stopped reporting bad luck and started reporting a defect in the instruction. Output that is well-formed and domain-plausible and still answers the wrong question will keep answering the wrong question, and retrying is the wrong response. The prompt is not giving the model what it needs to succeed, and no number of re-rolls against the same prompt will manufacture the missing information. The correct action is to escalate and fix the instruction, not to keep spending the budget. Treating repeated semantic failure as a transient error to retry through is how a pipeline burns its entire budget on every hard case and still delivers the wrong answer.

For failures that look transient but keep landing in the same spot, deliberately injecting some variance can break the rut: nudging the sampling temperature up on later attempts, or rephrasing the instruction so the request is unchanged but the wording it arrives in is not. Either one knocks the model off the groove it keeps falling back into. Both carry a caveat. Piling on retries is not costless improvement. Across a long chain, an answer can go on satisfying every structural and format check while its meaning slides further from the target on each pass, until the loop settles on something that validates and still misleads. The budget guards against that slide as much as against runaway cost.

And when the budget is exhausted, the loop has to do something explicit. The dangerous move is to silently return an empty or default result, because an unmarked empty result is indistinguishable, to everything downstream, from a real answer that happens to be empty. Exhaustion has to hand the item somewhere a human or another system will see it, flagged for review or wrapped in an error the caller is built to catch. The escalation path is part of the architecture, defined before deployment, not an afterthought discovered during the first runaway loop.

Grounding turns generation back into retrieval

The validation loop catches fabrication after it happens. Grounding reduces how often it happens in the first place, and it is the single most effective lever against confabulation in an extraction pipeline.

The mechanism is direct. Place the specific source document or retrieved passage in the prompt and instruct the model to draw only from that text, never from general knowledge. That instruction changes what the task is. Left to answer from training alone, the model reaches for everything it has ever absorbed and closes any gap with the value that looks most likely, which is exactly why a field the source never mentioned still comes back filled. Bound to the passage, it has one permitted source, and a gap there is supposed to surface as silence rather than as a confident guess. The move from papering over gaps to reporting them is the whole game. A pipeline that invents a value when the evidence says nothing is untrustworthy in precisely the cases where trust matters most.

Grounding pairs naturally with a citation discipline. Requiring the model to point at the exact passage behind each field gives the source-fidelity check something to aim at: a specific span to compare the value against rather than the whole document. What exposes a field with nothing behind it is the check that resolves the pointer, not the requirement to emit one. Citations convert a grounding instruction from a hope into something that can be checked, and when the extraction spans several documents, tagging each one so a citation resolves back to its origin is what lets anyone verify the result at all.

Grounding is necessary but not sufficient, because fabrication is only one way an extraction goes wrong, and different faults call for different defenses. Restricting a field to a fixed set of permitted values makes an out-of-set answer impossible to emit, something grounding alone never guarantees. Pulling the sampling temperature down tightens variance and steadies consistency, but it leaves a grounding gap untouched: a colder model just fabricates the same wrong value more reliably. No one technique spans the whole failure space, because each answers a different failure, and a serious pipeline stacks several instead of leaning on one.

The loop is overhead, and overhead has to be justified

Every layer described here costs something. Domain and source-fidelity checks take engineering to write and maintain. Retries add latency and multiply model calls. A validation loop is not free, and applying the full apparatus everywhere is its own kind of mistake, so the decision to spend it should be deliberate.

The overhead earns its place wherever a wrong value does real damage after it leaves the pipeline. When the output drives a legal, financial, clinical, or compliance decision, one bad field can cost far more than every retry the loop will ever run, and the checking is cheap insurance against an expensive mistake. The same holds when extraction runs unattended across large document volumes: with no one reading each result, the loop is the only thing standing between a systematic fault and its silent spread into systems where undoing it later costs far more than catching it now. In both settings the economics are the same, because a silent error is expensive and the loop is what makes errors loud.

The case weakens as soon as a person is already watching. Where extraction is exploratory or interactive and a human eyes each result and can fix an obvious slip in seconds, the full apparatus can cost more in latency and calls than the errors it would have caught were worth. A light check with human correction is usually the right investment there. What sets the level is not how sophisticated the pipeline is but a single question: do extraction errors carry consequences that outweigh the cost of the machinery built to catch them? Honest engineering finds where that line actually falls instead of defaulting to either extreme.

The guarantee you can buy and the guarantee you have to earn

There are two separate guarantees a structured-extraction system needs, and they are earned in completely different ways. The guarantee that the output has the right shape is something you can buy outright at generation time: engage constrained decoding and structural error stops being possible. It arrives as a property of the mechanism, requires no checking after the fact, and does not erode as the schema grows. The guarantee that the output is correct cannot be bought this way, because no constraint on the form of an answer can reach the truth of its values. That guarantee has to be earned after generation, by checking the answer against the domain’s rules and against the evidence it claims to rest on, and by treating a failure as information to act on rather than a result to accept.

A validation loop is the machinery that earns the second guarantee. It exists because a schema-valid answer and a correct answer are different properties, and closing the gap between them is not a formatting concern but a design one. The pipelines that fail quietly in production are the ones that mistook the guarantee they bought for the guarantee they needed, validated the structure they had already secured, and shipped without ever checking whether the confident, well-formed values they were returning were true.