Data engineering in the AI era · intermediate · About 10 minutes
The AI assistant that quotes old prices
The sales team's AI assistant quoted a customer the old price, 6 weeks after it changed. The document is right. The assistant's copy of it is not.
You run the data pipelines behind the company's internal AI assistant.
What happened
Sales staff ask an internal AI assistant about products and prices. It answers from the company's documents: they are split into passages, each passage is turned into an embedding, and the embeddings sit in a vector database. A question finds the closest passages, and the model answers from them.
On 1 September the Pro plan went from $49 to $59 a seat. On 14 October a sales rep quoted a customer $49, straight from the assistant, and the deal was signed at that price.
The incident is made up. How things work in it is real, with sources at the end.
The evidence
Everything you need is here. Read it before you decide.
From sales rep and assistant, 14 October
Rep: What's the list price of the Pro plan per seat?
Assistant: The Pro plan is $49 per seat per month, billed annually. (Source: pricing)
| Rank | Document | Passage | Indexed on |
|---|---|---|---|
| 1 | pricing | Pro: $49 per seat per month, billed annually. | 2 June |
| 2 | pricing-2025 (archived) | Pro: $45 per seat per month, billed annually. | 2 June |
| 3 | plans-overview | Pro adds single sign-on and audit logs. | 2 June |
- 2 June 11:20INFOfull build started by hand: 3,214 documents (including /archive)
- 2 June 13:05INFO18,402 passages embedded and written, 0 deleted
- since thenno runs
pricing was edited on 1 September and now says $59. pricing-2025 sits in the archive
folder. Nobody remembers who set up the index or that it needs running.
Your decisions
Decision 1 of 3
Decision 1
Why did the assistant say $49?
Decision 2
What do you build?
Decision 3
What about the archived 2025 price list?
What went wrong
An assistant like this uses RAGRAGA way to answer from your own documents. A search finds the passages that match the question, and they are added to the prompt before the model writes.. It never reads the documents when you ask. It reads a copy: passages turned into embeddingsEmbeddingA list of numbers that stands for the meaning of a piece of text, an image or other data. Things with similar meaning get similar numbers, so a computer can find them. and stored in a vector databaseVector DatabaseA database built to store embeddings and quickly find the ones closest to a query, which is how AI apps search by meaning. when the index was built. The search finds the passages closest in meaning to the question and hands them to the model. If the copy is old, the answer is old, and it arrives with a source attached, which makes it more convincing, not less.
That makes the index a data pipeline like any other, and the usual questions apply. When does it run? What happens when a document changes or is deleted? What is in scope? Here the answers were: once, by hand; nothing; and everything, archive included. Closeness in meaning has no sense of time, so an archived price list competes with the current one on equal terms.
Treat the index as a product with an owner, a schedule driven by changes, deletes that actually delete, and a freshness check that alerts when it falls behind the documents it copies.
Keep these
- A retrieval assistant answers from a copy of your documents. Know how old the copy is.
- Update on change, and delete the old passages. Appending alone keeps old versions in play.
- Search ranks by meaning, not date. Keep archived material out, or filter it.
The ideas behind it
Sources
Where each fact comes from
This page was last checked against these sources on 14 October 2026.
- 1Introducing Contextual Retrieval
Anthropic · 14 October 2026
RAG retrieves relevant information from a knowledge base and adds it to the prompt. The documents are split into chunks, an embedding model turns each chunk into a vector, a vector database finds the chunks closest in meaning to the query, and those are added to the prompt.