Writing the Model’s First Words Is Stronger Than Asking for Them

Every instruction you give a model about the shape of its output is a request. You describe the artifact you want, and the model, weighing your description against everything it has learned about how a helpful reply begins, produces something that usually matches. Usually is the operative word, and for a system whose next step is a parser rather than a person, usually is the whole problem. There is one control that does not work this way. You are allowed to write the beginning of the model’s answer yourself, and hand it back to the model as something already true. The format stops being a thing the model must be persuaded to adopt and becomes a fact it has to build on. That difference, between asking and committing, is the entire technique, and understanding where its authority ends is what separates a reliable use of it from a superstition.

The model continues a document rather than answering a question

The conversational framing of a model call is a convenience, not a description of the machinery. What actually happens is that a structured document, a sequence of turns with roles attached, is assembled and handed to a model whose only capability is to extend it. The model does not receive a question and formulate a reply. It receives a partially written transcript and predicts what comes next. When the transcript ends after a user turn, the region the model is extending happens to be empty, so it chooses how to open, and that choice looks like the model deciding how to answer you. It is really the model deciding how a document of this kind tends to continue.

Once you see the interaction that way, the technique writes itself. If the region the model extends is simply the end of the document, then you can put something there. You end the transcript not with the user’s turn but with the opening fragment of the model’s own, and the model continues from wherever your fragment stops. Nothing about the model has changed. Nothing was enabled and nothing was configured. You moved the cursor, and the model did what it always does, which is carry on from the cursor.

That explains why the mechanism has no configuration surface and needs none. It is simply the interface used deliberately rather than by default. Any interface that lets you author the turns and end the document on the model’s own lets you author the start of that turn, because the model has no way to distinguish text you wrote in that slot from text it wrote there itself. Both are simply prior context. That indistinguishability is the source of all the power here, and all of the danger.

The first token collapses the space of possible answers

The strange part of prefilling is how much a very small prefix accomplishes. A single opening brace changes the character of an entire response. The reason is that the opening of a reply is the moment at which the model resolves the largest amount of ambiguity about what kind of artifact it is producing. Before the first token exists, the response could still become almost anything: a paragraph of prose, an acknowledgment of the request, a caveat, a bulleted list, a table, an object. The first token eliminates nearly all of those simultaneously. After a brace, a continuation that drifts into conversational prose becomes wildly improbable, because nothing in the model’s experience of well-formed documents follows an opening brace with an apology.

The commitment then compounds. Every token the model emits becomes part of what it reads to choose the next one, so a response that has declared itself an object keeps finding that an object is the most natural thing to go on being. The prefix reaches well past the first token. It changes the document that the rest of the response is written against.

Compare that to the instructional route. An instruction to return only an object, with no preamble and no commentary, is competing against a deeply trained disposition toward conversational openings. The instruction usually wins. But when it loses, it loses in exactly the way that hurts most: the response is perfectly correct and preceded by a friendly sentence that your parser was never built to skip. The standard remedy is a scrap of code that hunts for the first brace and hopes for the best. It has to be written, maintained, and reasoned about forever, it hardcodes assumptions about a preamble whose shapes you have only partly seen, and its existence quietly concedes that nothing in the system actually enforces the format contract. Committing the opening token does not fight the model’s disposition toward a preamble. It removes the moment where the preamble could have been written. The failure that scrap of code exists to absorb stops occurring.

Format is the obvious application, but the same reasoning covers anything the opening establishes and the rest inherits. The language a response is written in is largely settled by its first few words. The register is set by them. A document whose first line is a heading tends to remain a structured document. In each case the leverage comes from the same place: you instantiate the property you want and let continuation carry it, instead of describing it and hoping.

What comes back is a continuation, not the whole artifact

There is a mechanical consequence of the prefix being yours that trips up systems on the first day and then intermittently forever. The model returns what it generated. It does not return what you wrote. If you committed an opening brace, the response you receive begins after that brace, and handing it directly to a parser produces a syntax error on a document that is, in every meaningful sense, correct. The artifact is the concatenation of your prefix and the model’s continuation, and reassembling it is your job because you are the one who split it.

This is trivial to get right and easy to get wrong later, which is the worse property. The reassembly lives in one place and the prefix lives in another, so a change to either without the other produces a class of bug that is invisible in review and obvious in production. Treat the prefix and the reassembly as a single unit of code, not as two independent decisions, and the problem goes away.

The other end of the artifact deserves the same scrutiny. A prefix constrains where generation starts and says nothing at all about where it stops. A model that has produced a complete object may keep going: a closing remark, an explanation of the fields it just emitted, a second object. Committing the opening buys you nothing at the tail. Whatever stopping discipline your output needs, whether a stop condition on the call or a parser tolerant of trailing content, is a separate design decision, and the confidence a clean prefix inspires is precisely what makes teams forget to make it.

The commitment binds the opening and only suggests the rest

Here is the boundary that matters most, and the one most often blurred. A prefix guarantees exactly one thing: how the response begins. It guarantees nothing about how the response proceeds. The influence of the prefix is real, and it thins with distance. By the time the model is deep inside a nested array, the token you wrote is a long way behind it, and everything the schema actually cares about, whether the required fields arrived, whether the types are what you declared, whether the object ever closes, is back in the hands of disposition.

So the correct mental model is that a prefix is an anchor, not a grammar. It fixes one point absolutely and everything downstream by inference. If what your system needs is conformance, meaning the output is guaranteed to carry the field names, the types, and the required fields your schema declares, then it needs a mechanism that constrains every token rather than the first one. That mechanism exists and works on a different principle. The schema is turned into a grammar that governs decoding itself, so the only continuations available to the model at any point are the ones that keep the output conformant. Under that mechanism there is no sampling path that leads to a violating output, so the question of likelihood never arises.

Three mechanisms are in play, and each acts somewhere different. Instructions describe intent, and they are what you use when the consumer is a human and small deviations are harmless. A committed prefix eliminates the class of failures that live in the opening, and it costs nothing but a few tokens. Grammar-constrained generation is what you use when the shape of the output is a contract that downstream code depends on. These are not rungs on a ladder to climb as far as your nerve allows. They act at different levels, and the failure is not choosing a weak mechanism, it is choosing one that does not act where the requirement lives. A system that genuinely needs schema conformance and reaches for a brace instead has confused an anchor for a guarantee, which is the most expensive misunderstanding available in this whole area, because it fails so rarely that the failure arrives with no supporting infrastructure around it and no one on the team expecting it.

A committed opening can foreclose the answers you needed

Every prefix is a decision that certain responses will not be produced. That is what makes it useful, and it is also its most underrated hazard, because the responses it forecloses are not always the ones you were trying to eliminate.

Consider what a system needs from an extraction endpoint. It needs the fields, obviously. It also needs, on some inputs, an honest statement that the fields are not there: the document was the wrong kind, the value was absent, the input was too corrupted to read. If your instructions leave room for the model to say so in plain language, and your prefix commits an opening brace, you have made that answer structurally impossible. The model must now produce an object. It cannot decline in prose because prose is no longer reachable. What it produces instead is an object, which is to say it produces the fields, whether or not the input contained them. You built a machine that cannot report absence, and then you are surprised when it reports something else.

The failure generalizes to any conflict between the prefix and the instructions. The prefix always wins the opening, so any instruction describing a differently shaped response is quietly overridden and the system behaves in a way that matches neither what the prompt says nor what the prefix implies. The instructions and the prefix have to agree, and agreement means more than not contradicting each other. It means every response the system legitimately needs, including the uncomfortable ones, must be expressible inside the shape the prefix committed to. In practice that pushes the error path into the schema, where it belongs: a nullable value, an explicit status, a confidence marker, a field whose presence means the model found nothing. Once absence has a representation inside the committed format, the format stops being a trap. The question to ask before adding any prefix is simply which legitimate answers this prefix makes unsayable, and it is a question with an uncomfortable answer more often than people expect.

A prefix should assert format, not content

The natural extension of a working technique is to use more of it, and here the extension goes wrong quickly. A prefix that grows past structure and into substance stops steering the model and starts writing the answer.

The line is clean. A brace, a tag, a heading, the first characters of a target language: these commit the model to a form and leave the content entirely to the model. But a prefix that opens the object and also fills in the first field has asserted a fact. The model will now build a response consistent with an assertion it did not make and cannot easily contradict, because contradicting the earlier part of your own document is exactly the behavior all that training pressure works against. You have not constrained the answer, you have prejudiced it, and you have done so in the one place in the request where the model has no capacity to disagree with you. Long prefixes also blur the handoff. The model has to work out where your fragment ends and its continuation begins, and a fragment that stops mid-sentence or mid-thought invites an awkward resumption that no amount of instruction fixes.

Keep the prefix to the smallest fragment that establishes the form. Then look at what the model actually writes next, because the resumption is a property of the exact fragment you chose and not of the idea of a short fragment, and a fragment that reads unambiguously to you can still land the model in a strange place.

The prefix belongs to the system, never to the request

Everything above is a design concern. This one is a security boundary, and it follows directly from the mechanism rather than from any policy.

The model cannot tell the difference between text it wrote and text you placed in its turn. That indistinguishability is what makes the prefix authoritative. It also means that whoever controls the prefix controls the model’s commitments more directly than any instruction in the system prompt can undo. A prefix can commit the model to a continuation it would otherwise have declined to produce. The model’s judgment was not argued away. The moment where that judgment would have been exercised, the choice of how to begin, was simply taken from it. Directing the mechanism at governance behavior rather than at output format is not a clever discovery about the model. It is a misuse of a control surface, and treating it as a general workaround for outputs a system was designed not to produce is how a formatting device turns into a liability.

The system-design consequence is the part that must not be missed. The prefix slot at the end of the document is a privileged control surface, and it has to be authored by the system, always, with no path by which request data reaches it. Any templating that interpolates user-supplied text into the prefix hands a caller the strongest steering primitive in the interface. That is the same category of error as letting user input become executable structure anywhere else in a system: data granted the authority reserved for the control plane. It is easy to introduce without noticing, because building the prefix from a user-selected format, a user-chosen language, or a user-supplied heading feels like configuration rather than injection. Treat the prefix as code, not as content, and keep the boundary between them explicit.

Most calls do not need one, and that is the point

A technique this effective invites overuse, and every prefix you add is a standing obligation. It is one more thing that has to stay consistent with the instructions and the schema as both of those evolve, and the three of them evolve at different times and usually in different hands. It quietly removes error paths that nobody remembered to preserve. And it couples application logic to an interface behavior that not every serving path exposes, which turns a change of provider or gateway into a change of your code.

There is one test, and it is narrow. Ask who is on the other end of the response, and whether anything they do depends on how it opens. If a parser, a router, a datastore, or any code that will fail or misbehave on an unexpected opening is waiting there, then the opening carries weight and committing it is close to free insurance. If a person is waiting there, or code with real tolerance for variation, then the instructions were always sufficient, a stray introductory sentence in front of a correct answer costs the reader nothing to skip, and the prefix is one more moving part maintained for the sake of appearing careful.

One moment of absolute control

Everything true about prefilling follows from a single fact about what a model does, which is extend a document. If the model extends a document, then the document’s final state before generation is the most authoritative thing in the request, more authoritative than any instruction, because it is not a description of the desired output. It is the output, already begun. That is why a few characters outweigh a paragraph of careful formatting instructions, and it is also why those few characters do not reach as far as their strength at the opening suggests they might.

Design to the actual shape of that authority. It is total at the first token and advisory thereafter, so use it to eliminate the failures that live in the opening and use a grammar-level mechanism for the ones that live in the structure. It forecloses alternatives absolutely, so make certain that every answer the system legitimately needs, including the admission that there is no answer, can still be expressed in the shape you committed to. It cannot distinguish your text from its own, so keep the request’s data out of it. And because it is a control worth having only where the opening actually carries weight, spend it on the paths where code is waiting on the other end, and leave the rest of the system to the instructions that were serving it perfectly well.

The prefix is the one place in a model call where you stop describing what you want and simply write it. Used for what it is, it buys more reliability per character than anything else in the request. Mistaken for a guarantee, it is a failure you have arranged to meet alone.