What is a sentence embedding, and how is it different from a word embedding?

A sentence embedding turns a whole piece of text, a sentence, a question or a paragraph, into one list of numbers. Texts that mean the same thing get lists that sit close together, so "How long until I get my money back?" lands near "We refund within 5 working days" and far from "Stand mixers have a 5-year warranty on the motor".

It is the kind of EmbeddingEmbeddingA list of numbers that stands for the meaning of a piece of text, an image or other data. Things with similar meaning get similar numbers, so a computer can find them. that search, clustering and RAGRAGA way to answer from your own documents. A search finds the passages that match the question, and they are added to the prompt before the model writes. run on. A search embeds every passage once, embeds each question as it arrives, and returns the passages whose lists are closest, usually measured by cosine similarityCosine SimilarityA measure of how closely 2 vectors point the same way, from 1 for the same direction to −1 for opposite; with embeddings, a higher score means closer in meaning..

Words versus sentences

Older word embeddingsWord EmbeddingAn embedding specifically representing individual words, capturing semantic relationships. gave each word one fixed list. To get a sentence you averaged its words, which loses the order and the context: "the dog bit the man" and "the man bit the dog" come out the same. A sentence embedding model reads the whole text at once, with a TransformerTransformerThe design behind almost every modern language model. It uses attention to let each word in a passage look at every other word, all at once., and is trained so that the final list stands for what the text means.

Why they made search fast

Before sentence embeddings, the most accurate way to compare 2 texts was to give both to a model like BERT at once. That works for one pair, but comparing a question with a million passages means a million model runs. The Sentence-BERT paper in 2019 put a number on it: finding the most similar pair among 10,000 sentences took about 65 hours that way, and about 5 seconds when each sentence was embedded once and the embeddings compared.

That is the trade every search system makes. Embedding each text separately, a bi-encoder, is fast and can be done ahead of time. Reading 2 texts together, a cross-encoder, is more accurate and too slow for a whole library, so it is used afterwards on a shortlist, a step called RerankingRerankingA second search step that reads each candidate passage together with the question and scores how well it answers, so the best few of a fast first search go to the model in the right order..

Using one

Hosted models are an API call: Voyage AI, which Anthropic's documentation points to since Anthropic makes none, OpenAI and Google all sell them, and the embedding cost tool compares what each charges. Open models run on your machine through the Sentence Transformers library:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("BAAI/bge-small-en-v1.5")
vectors = model.encode(["We refund within 5 working days.", "Stand mixers have a 5-year motor warranty."],
                       normalize_embeddings=True)
question = model.encode("Represent this sentence for searching relevant passages: When do I get my money back?",
                        normalize_embeddings=True)
print(vectors @ question)  # the refund sentence scores higher

2 details matter for search. Many models expect the question and the passages to be marked differently: bge models take a line before the question only, and Voyage has an input_type of query or document. And the lists from 2 different models cannot be compared, so changing the model means embedding every passage again.