What is "embedding" in language models? How are embeddings utilized in advanced prompt engineering techniques?

An embedding is a list of numbers (a vector) that represents the meaning of a piece of text, so that texts with similar meanings have vectors close together. Embedding models create them, and in prompt engineering they are used mainly to find the right content to put into a prompt: relevant documents, similar examples or past conversations.

How embeddings work

• An embedding model turns text such as "How do I reset my password?" into a vector of hundreds or thousands of numbers.
• Distance between vectors, usually cosine similarity, measures how related two texts are. "I forgot my login" lands near the password question even though they share no words.
• Inside a language model, every token also has an embedding, and its layers build richer contextual representations; dedicated embedding models produce one vector for a whole sentence or passage, optimised for search and comparison.

Uses in advanced prompt engineering

1. Retrieval-augmented generation (RAG) Split documents into chunks, embed and store them in a vector database or vector index. For each question, embed it, find the closest chunks and put them into the prompt as context. The model answers from current, specific sources.

2. Dynamic few-shot examples Keep a library of good input and output examples. For each request, choose the examples most similar to the input, so the prompt shows the model the most relevant cases.

3. Semantic caching If a new question is nearly identical to one answered before, reuse the stored answer instead of calling the model.

4. Routing and classification Compare a request's embedding with embeddings of categories or sample requests to send it to the right prompt, tool or team.

5. Memory for assistants and agents Store summaries of past conversations or notes as embeddings and retrieve the relevant ones later.

6. Evaluation and analysis Cluster user questions to find common topics, detect duplicates, or measure how close an answer is to a reference.

Practical points

• Use the same embedding model for documents and queries, and re-embed everything if you change models.
• Chunk size matters: too large dilutes meaning, too small loses context.
• Combine embedding search with keyword search (hybrid search) and a reranker for better results, especially for names, codes and exact terms.
• Choose a model suited to your languages and domain; some providers don't offer embeddings, so applications often use a separate embedding provider.