Base Models vs Chat Models

A pretrained model continues text; it does not answer questions. What instruction tuning adds, and why the assistant persona is a trained layer.

Take a model that has learned language from enormous amounts of text and ask it a question. You might not get an answer.

You might get more questions. Or a multiple-choice list. Or a “Frequently Asked Questions” heading followed by unrelated entries.

The model is not broken. It is doing exactly what it was trained for — continuing the text — and in its training data, a question is very often followed by other questions rather than an answer.

Base models

A base model has had pretraining only: predict the next token across a vast corpus. That gives it grammar, facts, reasoning patterns, code, and style.

What it does not give it is any notion that you are a person asking for something and it should help. There is no request-response frame. There is only text, and the question of what plausibly follows.

So base models behave like extremely capable autocomplete:

• Prompt The three main causes of the French Revolution were → a solid continuation
• Prompt What caused the French Revolution? → possibly a list of related exam questions

They are steerable, but only by making the desired output the natural continuation. Write the beginning of the answer and the model finishes it. This is a real skill, and it is not how anyone wants to work.

What instruction tuning adds

Turning a base model into an assistant takes additional training stages. The details vary by lab, but the shape is consistent.

Supervised fine-tuning. Train on many examples of instruction-and-good-response pairs. The model learns the format: input framed as a request, output framed as a helpful reply. This alone transforms behavior.

Preference training. Show the model pairs of responses where humans indicated a preference, and train it toward the preferred kind. This shapes tone, refusal behavior, hedging, formatting habits, and length. The common name is RLHF — reinforcement learning from human feedback — though several variants exist.

The result is an instruct or chat model. Same underlying architecture, same pretrained knowledge, different surface behavior.

What this means practically

The assistant is a layer, not an essence. The cheerful helpfulness is trained behavior sitting on a next-token predictor. Underneath, it is still predicting tokens. Nothing about the persona indicates the content is reliable.

Instruction tuning can cost raw capability. Aligning a model toward safe, helpful, predictable output sometimes narrows what it will attempt. This is a real tradeoff, occasionally visible on creative or unusual tasks, and it is why some practitioners still prefer base models for certain generation work.

Chat models expect a specific format. Instruct models were trained on a particular message structure — roles, turn markers, delimiters. APIs handle this for you, but it explains why a system prompt has a distinct effect from the same text pasted into a user message: they occupy different structural slots the model learned to treat differently.

Refusals come from this stage. A base model has no concept of declining. Refusal behavior is trained in during preference training, which is why it can feel inconsistent — it is learned tendency, not a rule engine.

Which one you are using

Almost certainly an instruct model. Anything you access through a chat interface, and nearly every default API model, is instruction-tuned.

Base models are usually labeled explicitly — often just the model name where the tuned version carries a suffix like -instruct or -chat. If you are working with open-weight models, check: downloading the base version and expecting chat behavior is a common early stumble.

What to remember

• A base model continues text; it has no request-response frame.
• Instruction tuning adds that frame via supervised fine-tuning plus preference training.
• The assistant persona is a trained surface over a token predictor — helpfulness is not evidence of accuracy.
• Alignment can trade away some raw capability, and refusal behavior originates here.

Next: Where Does the Vocabulary Come From? — back down to the input layer.