What is vector search?

Search a help centre for "can't sign in" and a keyword search misses the article titled "Resetting your password". The words do not match, though the meaning does. Vector search finds it, because it compares meaning: every text is turned into a list of numbers, and texts that mean similar things get similar numbers.

How it works

1. Embed the collection. An EmbeddingEmbeddingA list of numbers that stands for the meaning of a piece of text, an image or other data. Things with similar meaning get similar numbers, so a computer can find them. model turns each document, or each chunk of one, into a vector: a list of a few hundred to a few thousand numbers.
2. Embed the query with the same model.
3. Find the nearest vectors. Measure how close the query's vector is to each stored vector, usually with cosine similarityCosine SimilarityA measure of how closely 2 vectors point the same way, from 1 for the same direction to −1 for opposite; with embeddings, a higher score means closer in meaning., and return the closest few.

That third step is the "search". It has no idea about words at all. It only measures distances between points.

Exact or approximate

Comparing the query with every stored vector gives the exact answer, and for tens of thousands of vectors it is fast enough. pgvector in Postgres does exactly this until you add an index.

At millions of vectors that becomes too slow, so most systems build an index for approximate nearest neighbour searchApproximate Nearest Neighbor SearchFinding the stored vectors closest to a query without comparing it with every one, by using an index that may now and then miss the true closest in exchange for far faster searches.. HNSWHNSWHierarchical Navigable Small World: a vector index that links each vector to its near neighbours in layers of graphs, so a search hops from coarse to fine and checks only a small share of the vectors., the most common, links each vector to its near neighbours in a layered graph and walks the graph towards the query, which takes roughly logarithmic time instead of a full scan. The catch is in the word approximate: it may miss a few true nearest neighbours. The share it finds is called recall, and index settings trade recall against speed.

Where it is used

• Retrieval-augmented generationRAGA way to answer from your own documents. A search finds the passages that match the question, and they are added to the prompt before the model writes., finding the passages an AI model should read before it answers.
• Search boxes that should understand paraphrases and other languages.
• "More like this" recommendations, duplicate detection, clustering.

Where it falls short

Vector search is weak at exact matches: a product code, an error number, a person's name. The embedding of "error 0x80070005" is not reliably close to the one document that contains it. That is why many systems run hybrid searchHybrid SearchRunning a keyword search (usually BM25) and a vector search for the same query and merging the 2 ranked lists, so exact terms like a product code and loose wording like "money back" are both found., vector search and keyword search together, and merge the results. It also only finds what is similar to the question. If the answer is phrased very differently, a rerankingRerankingA second search step that reads each candidate passage together with the question and scores how well it answers, so the best few of a fast first search go to the model in the right order. step or a rewritten query helps.

Sizing it

Each vector costs its length times the bytes per number: 1,024 numbers at 4 bytes is 4 KB per chunk, before the index. The vector database size calculator works this out for a whole collection.