How do you choose an embedding model for a retrieval system?

Shortlist a few models using public benchmarks such as MTEB, filtered by what you need (languages, domain, input length, vector size, price, and where it can run), then test them on your own documents and questions, measuring recall@k. Pick the cheapest model that reaches your quality bar, and record which model made every vector, because changing it later means re-embedding everything.

Start from public benchmarks, but do not stop there

The Massive Text Embedding Benchmark (MTEB) covers 8 kinds of embedding task across 58 datasets and 112 languages. Its authors found that no single embedding method was best at every task. That is the key lesson: look at the retrieval scores, in your languages, and expect your own data to rank models differently again.

What to filter on

• Task fit: retrieval scores, not clustering or classification.
• Languages: multilingual models if users ask in one language and documents are in another.
• Domain: some models are trained for code, law, finance or medicine.
• Input length: the model's token limit decides your largest chunk.
• Vector size: longer vectors can be more accurate but cost more storage, memory and search time. Some models let you shorten vectors with a small loss.
• Query vs document input: some models expect a different input type or prefix for queries and documents; get this wrong and quality drops quietly.
• Hosting and data: a hosted API (check its data terms and region) or a model you run yourself.
• Price and speed: for the initial embedding of the whole corpus and for every query.

Test on your data

1. Build 100 or more real questions, each labelled with the passages that answer it.
2. Embed the corpus with each candidate, using the same chunking.
3. Measure recall@k and ranking quality, overall and per category (product names, long questions, other languages).
4. Add hybrid search and reranking to the comparison: a cheaper embedding plus a reranker can beat an expensive embedding alone.

Plan for change

• Store the model name and version with each vector.
• Changing the model means re-embedding the whole corpus; build the new index alongside the old one, evaluate, then switch.
• Vectors from different models cannot be mixed in one index.

What interviewers want: benchmarks to shortlist, your own evaluation to decide, and awareness of the hidden costs (re-embedding, storage) of the choice.