# What is RAG (retrieval-augmented generation)?

A way to answer from your own documents. A search finds the passages that match the question, and they are added to the prompt before the model writes.

- Page: https://tecyfy.com/concepts/rag
- Subject: Data engineering in the AI era (https://tecyfy.com/concepts)
- Level: intermediate
- Also called: Retrieval-Augmented Generation
- Last checked: 2026-10-14

A model knows only what was in its training text, up to its [knowledge cutoff](https://tecyfy.com/concepts/knowledge-cutoff). It has never seen your company's handbook, last month's tickets or this morning's price list. RAG, short for retrieval-augmented generation, is the usual way to give it that knowledge at the moment it answers, without training it again.

The name comes from a 2020 research paper by Patrick Lewis and colleagues. It paired a language model with a searchable index of Wikipedia, and the answers came out more specific and more factual than from the model alone.

## How it works

The work happens in 2 stages.

**Before any question.** Your documents are split into small chunks. An [Embedding](https://tecyfy.com/concepts/embedding) model turns each chunk into a list of numbers that stands for its meaning, and the lists go into a [vector database](https://tecyfy.com/concepts/vector-database).

**When a question comes in.** The question is turned into numbers the same way. The database finds the chunks closest to it in meaning. Those chunks are pasted into the prompt next to the question, and the model writes its answer from them.

The model itself is unchanged. It is reading an open book that someone handed it a moment ago.

## Where it goes wrong

The answer can only be as good as the passages the search found. A chunk cut from the middle of a document can lose the thing that makes it findable, such as which product or which year it is about. Anthropic found that adding a line of context to each chunk before indexing cut failed searches by 49%, and by 67% when a second step re-ranked the results.

When the search misses, the model may still answer, from memory, with confidence. So a good RAG app shows its sources, and you should click them.

## When you do not need it

If all your documents fit comfortably in the [context window](https://tecyfy.com/concepts/context-window), you can skip the search and give the model everything. Anthropic's rule of thumb is under about 200,000 tokens. RAG earns its keep when the library is far bigger than any prompt.

## Related

- [Embedding](https://tecyfy.com/concepts/embedding): A list of numbers that stands for the meaning of a piece of text, an image or other data. Things with similar meaning get similar numbers, so a computer can find them.
- [Vector Database](https://tecyfy.com/concepts/vector-database): A database built to store embeddings and quickly find the ones closest to a query, which is how AI apps search by meaning.
- [Vector Search](https://tecyfy.com/concepts#vector-search): Finding items whose embeddings are most similar to a query vector.
- [Context Window](https://tecyfy.com/concepts/context-window): Everything a model can see at once while it writes an answer, counted in tokens. When the window is full, something has to be left out.
- [Hallucination](https://tecyfy.com/concepts/hallucination): When a model states something false as if it were true, such as a wrong date, a made-up quote or a source that does not exist.

## Sources

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://arxiv.org/abs/2005.11401), Patrick Lewis and others, arXiv, 2020-05-22. RAG models combine pre-trained parametric memory (a seq2seq model) with non-parametric memory (a dense vector index of Wikipedia reached through a neural retriever). They produced more specific, diverse and factual language than a parametric-only baseline.
- [Introducing Contextual Retrieval](https://www.anthropic.com/news/contextual-retrieval), Anthropic, 2024-09-19. RAG retrieves relevant information from a knowledge base and adds it to the prompt. The documents are split into chunks, an embedding model turns each chunk into a vector, a vector database finds the chunks closest in meaning to the query, and those are added to the prompt. Chunks can lose the context they need to be found. A knowledge base under 200,000 tokens can go into the prompt whole. Contextual Retrieval cut failed retrievals by 49%, and by 67% with reranking.