1.4 Base Model vs. Instruction-Tuned
1.4 — From Base Model to Assistant: Instruction-Tuning at a Glance
Learning objective: understand at a high level why raw pretrained models don’t behave like chat assistants out of the box, and what changes that.
A model trained purely to predict the next token from internet-scale text (a “base model”) is good at continuing text plausibly, but it has no built-in tendency to follow instructions, refuse harmful requests, or hold a conversational back-and-forth — it just continues whatever pattern it sees. The assistant behavior you actually interact with is the result of additional training stages layered on top:
flowchart LR
A["Pretraining\n(next-token prediction\non internet-scale text)"] --> B["Base model\n(completes text plausibly,\nno 'assistant' behavior)"]
B --> C["Instruction tuning\n(fine-tune on instruction →\ngood-response examples)"]
C --> D["Preference / alignment training\n(e.g. RLHF — tuned on\nhuman/model feedback)"]
D --> E["Assistant model\n(follows instructions,\nrefuses, holds a conversation)"]
- Instruction tuning: fine-tuning on examples of instruction → good response, teaching the model the “assistant” behavior pattern.
- Preference/alignment training (e.g., RLHF or similar methods): further tuning using human or model feedback on which responses are preferred, shaping helpfulness, tone, and refusal behavior.
Why this shows up as a behavioral difference
A base model given the text “How do I pick a good lock?” simply continues the most statistically likely text after that string — which, depending on what was in its training data, could be a locksmith’s tutorial, a forum thread, or fiction, with no particular judgment applied. An instruction-tuned, aligned assistant model given the same text as a user instruction reasons about it as a request from a person, and responds according to whatever behavior its alignment training shaped — answering helpfully, adding caveats, or declining, depending on context and how it was tuned.
You don’t need the training mechanics to build with these models, but knowing this distinction explains two things you’ll hit constantly:
- Why “jailbreaks” exist at all — they’re attempts to push a model back toward its raw, unaligned completion behavior, effectively asking it to respond as a base model would rather than as its assistant training shaped it to.
- Why the same underlying model can behave very differently depending on how a provider has tuned it — two products built on similar underlying pretraining can have very different assistant behavior, tone, and refusal patterns purely from differences in instruction tuning and alignment.
This is a direct preview of Track 3’s core subject matter — nearly every attack technique covered there is, at some level, an attempt to exploit the gap between “what the base model would say” and “what the alignment training tries to constrain it to say.”
Module 1 Quiz
See quiz.md for the check-for-understanding questions.