# What is cosine similarity, and why is it used to compare embeddings?

A measure of how closely 2 vectors point the same way, from 1 for the same direction to −1 for opposite; with embeddings, a higher score means closer in meaning.

- Page: https://tecyfy.com/concepts/cosine-similarity
- Subject: Data engineering in the AI era (https://tecyfy.com/concepts)
- Level: beginner
- Also called: Cosine distance

Cosine similarity measures how closely 2 lists of numbers point the same way. It is a common way for AI search to compare a question with stored text: both are turned into [embeddings](https://tecyfy.com/concepts/embedding), and the pairs with the highest cosine similarity are the closest in meaning.

## The idea in one picture

Think of each embedding as an arrow from the same starting point. Two arrows pointing almost the same way score close to 1. Arrows at right angles score 0. Arrows pointing in opposite directions score −1. Only the direction counts, not how long the arrows are.

Turn and stretch the arrows below and watch which numbers change.

## The sum

Multiply the 2 lists number by number and add the results up: that is the dot product. Divide it by the length of each list, and you have the cosine similarity. scikit-learn's documentation puts it as the dot product of the 2 vectors after each is scaled to a length of 1, which is the cosine of the angle between them.

For A = [1, 2] and B = [2, 3]: the dot product is 1 × 2 + 2 × 3 = 8. The lengths are √5 and √13. So the cosine similarity is 8 ÷ (√5 × √13), about 0.99. The arrows point almost the same way.

## Cosine, dot product or distance?

There are 3 common ways to compare embeddings: cosine similarity, the dot product, and the straight-line (Euclidean) distance between the tips. When every vector has a length of 1, they all put the results in the same order. OpenAI says its embeddings are normalised to length 1, so the dot product gives the cosine similarity with less work, and cosine and Euclidean distance rank results identically. It also notes the choice of measure typically doesn't matter much.

It matters when vectors are not normalised. Then the dot product rewards long vectors, and the distance punishes them, while cosine ignores length altogether. Use the measure the embedding model's makers recommend, and build your [vector index](https://tecyfy.com/concepts/vector-database) for that same measure.

## Similarity and distance

Databases often store a *distance*, where smaller is closer, rather than a similarity. In pgvector, the `<=>` operator returns cosine distance, which is 1 minus the cosine similarity:

```sql
SELECT id, 1 - (embedding <=> '[3,1,2]') AS cosine_similarity
FROM items
ORDER BY embedding <=> '[3,1,2]'
LIMIT 5;
```

## What a score does not tell you

A score has no fixed meaning across models: the scores typical of related and unrelated texts differ from one model to the next, so a threshold such as "only keep matches above 0.8" has to be set for your model on your own data. Compare scores from the same model only: vectors from different models cannot be compared at all.

(An interactive demo goes with this on the page: https://tecyfy.com/concepts/cosine-similarity)

## Related

- [Embedding](https://tecyfy.com/concepts/embedding): A list of numbers that stands for the meaning of a piece of text, an image or other data. Things with similar meaning get similar numbers, so a computer can find them.
- [Vector Database](https://tecyfy.com/concepts/vector-database): A database built to store embeddings and quickly find the ones closest to a query, which is how AI apps search by meaning.
- [HNSW](https://tecyfy.com/concepts/hnsw): Hierarchical Navigable Small World: a vector index that links each vector to its near neighbours in layers of graphs, so a search hops from coarse to fine and checks only a small share of the vectors.
- [Approximate Nearest Neighbor Search](https://tecyfy.com/concepts/approximate-nearest-neighbor-search): Finding the stored vectors closest to a query without comparing it with every one, by using an index that may now and then miss the true closest in exchange for far faster searches.
- [RAG](https://tecyfy.com/concepts/rag): A way to answer from your own documents. A search finds the passages that match the question, and they are added to the prompt before the model writes.

## Sources

- [Pairwise metrics, Affinities and Kernels: cosine similarity](https://scikit-learn.org/stable/modules/metrics.html)
- [Vector embeddings](https://developers.openai.com/api/docs/guides/embeddings)
- [pgvector](https://github.com/pgvector/pgvector)