# What is instruction tuning, and how does it turn a base model into an assistant?

Fine-tuning a pre-trained model on many tasks written as instructions with good answers, so it follows a request instead of just continuing the text.

- Page: https://tecyfy.com/concepts/instruction-tuning
- Subject: How AI works (https://tecyfy.com/concepts)
- Level: beginner
- Also called: Instruction Fine-Tuning

A model fresh from pre-training is a text continuer. Give it "Write a short refund policy for a kitchen shop" and it was trained only to predict what text comes next, so it may treat the line as the start of a longer document and carry on writing that, rather than write the policy. Instruction tuning is the training step that teaches it to treat the text as a request and answer it.

## How it is done

Take a pre-trained [language model](https://tecyfy.com/concepts/large-language-model) and [fine-tune](https://tecyfy.com/concepts/fine-tuning) it on thousands of examples, each an instruction and a good response: "Translate this into French", "Summarise this article in 2 sentences", "Is this review positive or negative?". The model learns the general pattern of following an instruction, not only the tasks it saw.

That last point is what Google's FLAN paper showed in 2021. They tuned a 137 billion parameter model on over 60 tasks phrased as instructions and tested it on kinds of task it had never been tuned on. It beat the larger GPT-3, asked the same way without examples, on 20 of 25 of them.

## Where it sits

Instruction tuning on written demonstrations is also called [supervised fine-tuning](https://tecyfy.com/concepts#supervised-fine-tuning). In OpenAI's InstructGPT work it was the first of 2 steps: tune on answers written by people, then improve further with [RLHF](https://tecyfy.com/concepts/reinforcement-learning-from-human-feedback), where people rank the model's answers. Today's assistants all come out of a pipeline like this, which is why a model you call through an API answers questions and follows a [system prompt](https://tecyfy.com/concepts/system-prompt) rather than continuing your text.

For an engineer the practical point is that the model was trained to be helpful to whoever writes the instruction, and it sees every instruction in its context. That is useful when the instruction is yours and dangerous when it comes from a stranger's email.

## Related

- [Fine-Tuning](https://tecyfy.com/concepts/fine-tuning): Training an existing model a little further on your own examples so it picks up a task, a format or a style. Often a good prompt does the job first.
- [Supervised Fine‑Tuning](https://tecyfy.com/concepts#supervised-fine-tuning): Standard fine‑tuning on labeled input‑output pairs.
- [Reinforcement Learning from Human Feedback](https://tecyfy.com/concepts/reinforcement-learning-from-human-feedback): A training step where people rank a model's answers, a reward model learns those preferences, and the language model is tuned to score well on it.
- [Large Language Model](https://tecyfy.com/concepts/large-language-model): A model trained on a huge amount of text to predict the next token. Further training turns it into an assistant that follows instructions.
- [Alignment](https://tecyfy.com/concepts#alignment): Ensuring model objectives match human values and intent

## Sources

- [Finetuned Language Models Are Zero-Shot Learners](https://arxiv.org/abs/2109.01652)
- [Training language models to follow instructions with human feedback](https://arxiv.org/abs/2203.02155)