When the Standard Is Real but Unwritable, the Reviewer Is Another Model

Deterministic validation can only enforce standards you can state as a rule. A schema check, a range check, a check that two fields agree: each one closes a failure you were able to write down in advance. But a large share of what makes an output good cannot be written down that way. Is this summary faithful to what it summarizes? Does this argument actually hold together, or does it merely sound like it does? Is anything important missing? Is the tone right for the reader it is meant for? These are judgments, not assertions. No predicate returns true or false on them, because the standard they measure against lives in a reader’s head, not in a rule. When the standard is real but you cannot express it as a check, the thing you insert into the pipeline to enforce it is a second model call given one assignment: judge the first. That is multi-pass review, and both its power and its central hazard come from the same fact: you are asking a model to grade a model.

The distinction from a validation loop is precise. A deterministic check answers one question, does this output break a rule I was able to specify. That question closes off every failure you could anticipate and encode. It says nothing about the failures you can only recognize when you see them. Multi-pass review exists for exactly that residue: the quality that a competent reader would catch in a second pass but that no assertion in your codebase will ever flag.

The reviewer is a reader, not a second author

The pattern is easy to describe and easy to get subtly wrong. Generation happens once. Then a distinct call reads that output against a stated standard and reports where it falls short. Only if something is flagged does a further call rewrite the flagged material, and that rewrite is what ships. Two of the three calls author nothing; they read. The reviewer’s entire function is to weigh what already exists against a standard.

That separation is the whole point, and it is where the value comes from. A reviewer that is asked to evaluate does a different and easier job than a generator asked to produce. Generation is open-ended construction under many simultaneous constraints, holding tone and structure and accuracy and completeness in mind all at once while inventing content from nothing. Review is bounded discrimination: the answer is already on the page, and the task is to decide whether it satisfies a defined standard. Discrimination against a fixed target is a genuinely lighter cognitive load than construction against a blank one, which is why a second pass can catch what the first pass produced. The reviewer is not smarter than the generator. It is doing a smaller, more tractable task with the answer already in front of it.

This also explains what a reviewer model buys you that a human reviewer does not, and it is not accuracy. A human reviewer applies a standard inconsistently, drifts over a long day, and reads the tenth output more loosely than the first. A reviewer model prompted with a fixed checklist applies the same standard to the thousandth output as to the first, without fatigue and without drift. Consistency, not brilliance, is the property you are buying. The pattern turns an inconsistent, expensive, unscalable act of judgment into a consistent, cheap, scalable one, and that trade is what makes it worth building.

A model grading its own work is a compromised judge

The uncomfortable question sits underneath the whole pattern. If the same intelligence produced the error, how can it catch the error? Part of the answer is the generation-versus-review asymmetry above. The rest of the answer is where the pattern’s hardest design decision lives.

Using the same model for both generation and review is the cheap path, and for a whole class of checks it works. If the model produced a summary that is missing its required conclusion, a second pass asking specifically whether a conclusion is present will find it missing, because presence is not something the model has an opinion about. But the moment the check depends on knowledge, the same-model reviewer is captive to the same gaps that produced the error. A model that fabricated a figure because it did not know the real one will not flag the fabrication on review, because from the inside a confident guess and a known fact are indistinguishable. It does not lack the fact on generation and then possess it on review. The blind spot is shared across both passes, so the second pass sees nothing wrong. Worse, a model tends to read its own wording as deliberate and score itself kindly. That leniency is not a hazard you might introduce; with one model on both sides of the exchange it is structural.

A different model as reviewer, and especially a stronger one, breaks that correlation. It does not share the first model’s specific gaps, so a fabrication that the original model believes will strike the reviewer as unsupported. Independence is the thing you are paying the higher cost for, and it is worth precisely as much as the correlation it breaks.

Which reframes the self-versus-cross decision more usefully than treating it as a pure function of stakes. The right question is which class of failure you are guarding against. Completeness and format defects are uncorrelated with the model’s knowledge; any competent reader catches a missing section or malformed structure, so a same-model reviewer handles them cheaply and there is little to gain from a second model. Fidelity and accuracy defects are the opposite: they arise from exactly the knowledge gaps a same-model reviewer shares, so self-review is weakest precisely where it appears to be working. That is the pattern’s most dangerous illusion, a self-review pass that approves fabricated content because the reviewer holds the same false belief that produced it. When the check you care about is whether the values are true, correlated blind spots make an independent reviewer close to mandatory. Stakes tell you whether to review at all. The correlation of the failure with the model’s own knowledge tells you whether the reviewer can be the same model.

The reviewer is fallible in both directions

A deterministic check has a property worth naming because the reviewer model lacks it: a failing check is definitionally a real failure. The rule fired because the rule was violated. A model critic offers no such guarantee. It can miss a real fault, and it can also invent one, flagging a problem in an output that was correct. Both errors carry a cost, and the second is the one teams forget to design against.

A false positive from the reviewer triggers a revision the output did not need. The revision call dutifully “fixes” a problem that was not there, and a correct value gets rewritten into a worse one to satisfy a critique that should never have been raised. The reviewer is not an oracle handing down ground truth; it is a second fallible reader whose output is itself only as reliable as the model producing it. That has two direct consequences for how the pattern is built. The reviewer prompt has to constrain the critique to a defined standard rather than inviting open-ended fault-finding, because an unconstrained reviewer asked to find problems will find them whether or not they exist. And the revision step has to be scoped so that a spurious critique cannot rewrite the whole output, only the specific span the critique named. A reviewer that can only raise defined failures, feeding a reviser that can only touch what failed, keeps a wrong critique from spreading past the span it named. A reviewer given free rein to opine, feeding a reviser given free rein to rewrite, lets one bad judgment cascade through an output that was fine.

A reviser can only be as precise as its instructions

Almost every weak implementation of this pattern fails at the same point: the reviewer prompt. The instinctive version hands the reviewer an open brief, read it over and say what is wrong, and what comes back is mood rather than measurement. The complaints differ from one run to the next, and none is precise enough for the revision step to act on. Asked to find problems in general, the reviewer returns impressions, and impressions do not tell a reviser what to change.

The reliable version replaces the open request with an explicit checklist, each item phrased as a condition with a definite pass or fail. Not “is this good” but “is the required recommendation present,” “does every quoted quantity trace back to the input,” “is the register the one the brief asked for.” And the critique itself has to be structured, not prose: for each failed item, the check that failed, a short account of how it failed, and a specific proposed correction. That structure is not bureaucratic overhead. It is what makes the critique actionable, because the revision step can only be as precise as the critique that drives it. A reviser handed “this could be tightened” produces cosmetic churn. A reviser handed “the check for a conclusion failed; the output ends mid-analysis; add a concluding synthesis” has a defined target and hits it. The specificity of the fix is set entirely by the specificity of the fault report.

The checks themselves tend to fall into three families, and a complete reviewer prompt covers all three rather than defaulting to whichever is easiest. Completeness asks whether every required element is present, and it is the failure most easily missed on a quick human read of long output and most easily caught by a systematic reviewer. Fidelity asks whether the content is consistent with the input it was drawn from, whether a figure was invented or a source claim contradicted, and it is the family where correlated blind spots make an independent reviewer earn its cost. Form asks whether tone, voice, and structure match what was specified. A prompt that checks only the family it finds convenient gives the reviewer a partial picture and lets the other families through untouched.

Detection without repair changes nothing

Detecting a problem accomplishes nothing on its own; the revision pass is what turns a critique into a better output. The revision call is handed exactly two inputs, the output as produced and the itemized verdict on it, and returns a version that answers each named failure. This is the payoff for the discipline in the reviewer prompt. Specific failures with suggested corrections give the reviser clear targets; a vague critique yields a vague revision, and the loop spends a model call to move sideways.

The loop does not close until the revision is checked the same way the original was. Re-running the checklist against the revised output is not optional, because a revision is itself generated content and can fail its own check, either by not fully resolving the original fault or by introducing a new one while fixing the old. A reviser told to repair a tone problem can quietly drop a required section in the process. So the revision re-enters review, and the pattern acquires the same shape as any correction loop: a bounded number of attempts, because a critique-and-revise cycle left uncapped can oscillate, each pass satisfying the last critique while tripping a new one, spending calls without converging. When the cap is reached and the output still fails, the item goes to a human rather than shipping unreviewed, for the same reason any exhausted loop escalates rather than guessing.

Multi-pass review is not a patch for a weak prompt

The most common misuse of the pattern is architectural, and it looks enough like diligence to escape notice. The tell is a reviewer bolted onto a generator nobody fixed. The output wobbles, so a second pass is added to catch the wobble, and now every call costs more, waits longer, and still ships a share of the same defects, because the thing that produced them was never touched. Review has been drafted to compensate for generation, and it compensates poorly and at a premium.

The correct sequence is the reverse. Drive the generator to a dependable floor first, so the output is usually right on the first pass. Then add review as a control over the variance that survives a prompt you can no longer improve. A second pass is a control on residual variability, not a way to avoid reducing that variability at the source. Reaching for a reviewer before fixing the generator is treating the symptom while paying to keep the disease. The reviewer’s job is to catch the residual failures a well-built prompt still occasionally produces, not to stand in for the prompt engineering that was never done.

The pass costs a call and a wait, so it has to buy something

None of this is free. A reviewer pass adds at least one model call and its latency, and a failed critique adds a revision call on top, and at high volume those multiply into real cost. Applying the full apparatus to every output is its own failure of judgment, the mirror image of applying none.

The pass earns its place where a defect does real damage after it leaves the pipeline. When a compliance ruling, a clinical judgment, or a financial posting rests on the content, a single missed error can cost far more than every review call the system will ever make, and in regulated settings an independent check is often less a quality preference than the precondition for anyone trusting the output. It earns its place for long-form output, where a dropped section slides past a fast read and a systematic checklist catches it. The case collapses in the opposite conditions. Rough drafts, idea generation, first-pass exploration: the reader is already going to rework these, so a review pass only adds cost and latency to output nobody was treating as final. High-volume, low-value generation is the same economics from the other side, where a review call on each of thousands of cheap outputs multiplies spend out of proportion to the harm any one error does. And where a human already vets the output before it matters, adding a model reviewer on top demands an honest answer about what the second pass actually contributes, rather than stacking machinery out of habit.

What decides the level is not how sophisticated the pipeline is. It is a single question about consequences: does a review pass catch errors whose cost outweighs the cost of the pass. Where the answer is yes, the pattern pays for itself many times over. Where it is no, it is overhead dressed as rigor.

Judgment-level quality needs a judge, and the judge has to be built with care

Structured output settles the shape of an answer, and deterministic validation settles the rules you could write down. Multi-pass review is what remains for the quality that is neither shape nor rule but judgment, the faithfulness, completeness, and fitness that a competent reader recognizes and no assertion captures. The pattern works by turning an inconsistent human act into a consistent mechanical one, applying the same standard to every output without drift.

But it works only when it is built against its own weaknesses rather than in ignorance of them. The reviewer has to be independent enough to break the blind spots that produced the error, which for accuracy and fidelity means a different model and not the one being graded. The critique has to be specific enough that a reviser can act on it, which means an explicit checklist and a structured verdict, not an invitation to find problems. And the reviewer has to be understood as fallible in both directions, which means constraining what it can flag and scoping what the reviser can touch, so that a wrong critique cannot cascade. A multi-pass system built without those constraints does not add quality control. It adds a second source of error, wrapped in the appearance of rigor, and charges you for the privilege. The systems that get real value from a second pass are the ones that treat the reviewer not as an oracle but as what it is, a second fallible reader whose consistency is its whole contribution, and design around exactly that.