# What is in-context learning, and how is it different from fine-tuning?

A model doing a new task from instructions and examples in its prompt alone, with no change to its weights; it forgets the task when the call ends.

- Page: https://tecyfy.com/concepts/in-context-learning
- Subject: How AI works (https://tecyfy.com/concepts)
- Level: intermediate
- Also called: ICL

In-context learning is a model picking up a task from what is in its prompt. Show it 3 emails sorted into categories, then a fourth, and it sorts the fourth the same way. Nothing about the model changes: its weights are exactly as they were, and on the next call, without the examples, it has forgotten the task.

## Why it was a surprise

Until about 2020 the way to teach a model a new task was [Fine-Tuning](https://tecyfy.com/concepts/fine-tuning): thousands of labelled examples and a training run. The GPT-3 paper showed that a large enough model could do many tasks from a description and a handful of examples given as plain text, with no training step at all, sometimes as well as fine-tuned models. It also listed tasks where this still fell short. That ability is the reason building on a model through an API is practical: you change its behaviour by changing the prompt.

## What the model learns, and what it does not

"Learning" is a generous word. A 2022 study found that on classification tasks the examples' labels could be randomly wrong with little loss; what the model takes from them is the set of possible answers, the kind of input and the format. In other words, examples mostly tell the model what job this is and what an answer looks like. They are a weaker way to teach a precise rule than stating the rule, and a weaker way to teach new facts than giving the model the facts.

## In-context learning or fine-tuning?

| | In-context learning | Fine-tuning |
| --- | --- | --- |
| Changes the model | No | Yes, its weights |
| Cost to change | Edit the prompt | A training run |
| Cost per call | The examples are input tokens every time | No examples needed |
| Lasts | One call | Until you train again |

Start with the prompt: it is quicker to change and easy to test. Put facts that change, like prices and policies, in the context through [retrieval](https://tecyfy.com/concepts/rag) rather than training them in. Consider fine-tuning when the examples needed are too many or too costly to send with every call. Everything in context also competes for the model's attention: Anthropic notes that recall degrades as the [context](https://tecyfy.com/concepts/context-window) grows, so more examples are not always better.

## Related

- [Few-shot Learning](https://tecyfy.com/concepts/few-shot-learning): Putting a few worked examples of the task in the prompt, so the model copies their format, labels and tone without any training.
- [Zero‑shot Learning](https://tecyfy.com/concepts#zero-shot-learning): Solving a task without any task‑specific examples
- [Fine-Tuning](https://tecyfy.com/concepts/fine-tuning): Training an existing model a little further on your own examples so it picks up a task, a format or a style. Often a good prompt does the job first.
- [Context Window](https://tecyfy.com/concepts/context-window): Everything a model can see at once while it writes an answer, counted in tokens. When the window is full, something has to be left out.
- [Prompt Engineering](https://tecyfy.com/concepts/prompt-engineering): Writing and testing the instructions, context and examples you give a model so it does a job reliably, and measuring each change instead of guessing.
- [RAG](https://tecyfy.com/concepts/rag): A way to answer from your own documents. A search finds the passages that match the question, and they are added to the prompt before the model writes.

## Sources

- [Language Models are Few-Shot Learners](https://arxiv.org/abs/2005.14165)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://arxiv.org/abs/2202.12837)
- [Context windows](https://platform.claude.com/docs/en/build-with-claude/context-windows)