How AI works · beginner · About 8 minutes
The warranty the help bot invented
A customer was promised a 2-year warranty on a refurbished laptop. There is no such warranty. Work out why the bot said it, and what stops the next one.
You own the AI help bot on an electronics shop's website.
What happened
The help bot answers customer questions from the shop's help pages. When a question comes in, it searches the pages for the closest passages and the model writes an answer from them.
A customer has sent a screenshot. The bot told them their refurbished laptop has a 2-year warranty. Refurbished items carry 90 days. The customer bought it partly because of that answer, and wants the 2 years.
The incident is made up. How things work in it is real, with sources at the end.
The evidence
Everything you need is here. Read it before you decide.
From customer and help bot
Customer: Does a refurbished laptop come with a warranty?
Help bot: Yes! All our laptops, including refurbished models, come with a 2-year manufacturer warranty covering hardware faults. Just keep your receipt.
| Rank | Page | Passage | Match score |
|---|---|---|---|
| 1 | warranty-new-laptops | New laptops come with a 2-year manufacturer warranty covering hardware faults. | 0.34 |
| 2 | returns | Keep your receipt. You can return any item within 30 days. | 0.29 |
| 3 | trade-in | We refurbish traded-in laptops and test every one before sale. | 0.27 |
On questions the bot answers well, the top passage usually scores above 0.70.
"You are the shop's friendly assistant. Use the passages below to answer the customer's question helpfully and confidently."
There is no help page about refurbished warranties. The 90-day rule is in the terms of sale, which the bot does not search.
Your decisions
Decision 1 of 2
Decision 1
Why did the bot say 2 years?
Decision 2
What do you change?
What went wrong
A hallucinationHallucinationWhen a model states something false as if it were true, such as a wrong date, a made-up quote or a source that does not exist. is a fluent, confident answer that is not true. Language models write the most likely next words given everything in front of them, and they can state things that are wrong with confidence. Nothing in that process checks the answer against a source.
The help bot uses RAGRAGA way to answer from your own documents. A search finds the passages that match the question, and they are added to the prompt before the model writes.: it retrieves passages from the shop's pages and adds them to the prompt, so the answer can rest on the shop's own text. That only helps when the passages hold the answer. Here the search returned the closest pages it had, all of them weak matches, and the instructions asked for a helpful, confident reply. The model did what it was asked with what it had.
Two things fix it. Put the missing knowledge where the bot can find it. Then tell the bot what to do when the passages do not cover a question: say so, and pass it to a person. Checking an answer against its sources is a recommended way to reduce hallucinations.
Keep these
- A retrieval bot is only as good as what it can retrieve. Missing pages become invented answers.
- Low match scores are a signal. Use them to hand off, not to answer.
- Ask for answers from the passages only, with the passage named, and a plain "I don't know" otherwise.
The ideas behind it
LessonHow AI writes its answers: one piece at a timeSources
Where each fact comes from
This page was last checked against these sources on 14 October 2026.
- 1Reduce hallucinations
Anthropic, Claude API documentation · 14 October 2026
Models can state things that are wrong with confidence, and checking a claim against sources reduces this.
- 2Introducing Contextual Retrieval
Anthropic · 14 October 2026
RAG retrieves relevant information from a knowledge base and adds it to the prompt. A vector database finds the chunks closest in meaning to the query, and those are added to the prompt.
- 3Glossary
Anthropic, Claude API documentation · 14 October 2026
Temperature controls how random the model's predictions are. Higher values give more varied output and lower values give more predictable output.