Walk me through a retrieval-augmented generation (RAG) pipeline, step by step. Where does it usually fail?

RAG has 2 halves. Indexing, done ahead: load documents, split them into chunks, embed each chunk, and store the vectors with their text. Answering, per question: embed the question, retrieve the closest chunks, optionally rerank them, and give the best ones to the model with an instruction to answer only from them and cite them. Most failures are retrieval failures: the right passage never reached the model, so the model answered from memory or said nothing useful.

1. The steps and their typical failure

2. How to tell retrieval from generation

Keep a set of questions with the passage that answers each. Measure recall at k: how often that passage is in the top k retrieved. If recall is low, no prompt will fix the answers. If recall is high and answers are still wrong, work on the prompt and the model.

3. The fixes that move recall most

• Hybrid search: add keyword (BM25) search beside vectors, so exact terms match. See Hybrid Search.
• Context on each chunk: prepend a line saying which document and section a chunk comes from. Anthropic measured that this alone cut failed retrievals by 35%, and by 49% with keyword search added.
• Reranking: retrieve 50, rerank, keep 5. See Reranking.
• Chunk size tuned on your data, not a default.

4. Guard the answer

Tell the model to answer only from the supplied passages, cite the passage ID, and say it does not know when they do not answer the question. Log questions with no good match: they show what your documents are missing.

See RAG and Vector Database.

What interviewers want: indexing and answering as separate halves, each step's failure mode, retrieval measured on its own with recall at k, hybrid search, chunk context and reranking as the strongest fixes, and grounded answers with citations.