Supervised fine-tuning: demonstrations of behavior
Examples of good responses shift a pretrained model toward the behavior an application needs.
In this lesson
Supervised fine-tuning trains on desired outputs; it is different from showing examples inside one prompt.
Teach by demonstration
A base language model predicts continuations. A useful assistant must also respond to instructions, choose an appropriate format, and recognize the boundaries of a conversation. Supervised fine-tuning, often shortened to SFT, supplies examples of desired input-output behavior and updates parameters to make those outputs more likely.
The 2022 InstructGPT study used human demonstrations as an initial post-training stage. The T0 research project explored supervised training across tasks written as prompts. These are distinct examples of instruction-oriented training, not evidence that every assistant shares a single recipe.
A familiar objective, a different selection of data
For a response example, token-level loss can reward the demonstrated answer conditioned on the prompt. A common arrangement excludes prompt tokens from the response loss, though implementations and training goals vary. The crucial point is that a gradient update occurs. The example changes the checkpoint rather than merely occupying its current context.
Dataset design is therefore behavior design. If examples always provide long answers, the model can learn that long answers are expected. If uncertain questions are answered confidently, the demonstrations teach that pattern too. A beautifully formatted answer can still be a poor training target when its reasoning or facts are wrong.
What demonstrations leave unspecified
A single target response does not describe every acceptable response. For a translation task, several phrasings may be equally good; for a support task, different clarifying questions may be useful. The data capture a selection of behavior, not a complete specification of quality. Preference comparisons can provide a complementary signal about alternatives.
Coverage matters. Training on direct questions alone may not teach robust behavior when the input contains misleading quotations or incomplete context. A practical dataset review asks which situations are represented, which groups or languages are absent, and whether the examples reward the intended action rather than a superficial stylistic cue.
Small adaptation, real tradeoffs
Fine-tuning can update all parameters or a selected subset. It may make a model more useful on the target distribution while changing other behaviors. “Fine” does not mean the effect is inherently harmless or the data requirements are negligible. Evaluation should include both the desired behavior and important capabilities that must remain intact.
In an original example, demonstrations ask for three-line summaries of short reports. A successful tuned model follows that structure on unseen reports. It has not thereby learned to retrieve reports from the web or to verify every claim in them. Those require other mechanisms and evidence, even if the final interface makes the experience feel unified.
Prompt example or training example?
Putting two examples in a request changes the input for that request. Adding those examples to a fine-tuning dataset and running optimizer updates changes model parameters. Both can influence behavior, but they have different persistence and costs.
Fine-tuning is just a long prompt saved permanently.
Fine-tuning changes weights through optimization. A prompt supplies context to otherwise fixed weights.
Take it with you
- SFT learns from demonstrated outputs.
- The training examples encode behavior choices.
- Evaluate retained capabilities as well as the new target behavior.
Check your understanding.
Choose an answer, then see the explanation. You can try again.
Practice only. Your answer is not stored or sent anywhere.
Preference learning: comparing alternatives
Continue when you’re ready. Marking this lesson as read is optional.
Sources & further reading
Primary references for this lesson. The explanation and examples above are original.
- Training language models to follow instructions with human feedback ↗Long Ouyang et al. · 2022-03-04 · checked 13 Nov 2026
- Multitask Prompted Training Enables Zero-Shot Task Generalization ↗Victor Sanh et al. · 2021-10-15 · checked 13 Nov 2026