# Embedding and retrieval models
An embedding and retrieval model is a neural encoder that maps a piece of text (and, in newer systems, an image, audio clip, or video) into a fixed-length vector so that semantically related items land close together in a shared space.[1](https://arxiv.org/html/2405.17428v1) • [2](https://arxiv.org/html/2605.27295) These [bi-encoder](https://www.edgepedia.org/bi-encoder) models are the retrieval layer behind semantic search, recommendation, and retrieval-augmented generation (RAG), where a foundation model's answers are conditioned on documents fetched by vector similarity.[2](https://arxiv.org/html/2605.27295) • [3](https://developer.nvidia.com/blog/best-in-class-multimodal-rag-how-the-llama-3-2-nemo-retriever-embedding-model-boosts-pipeline-accuracy/)
| Fact | Value |
|---|---|
| Core mechanism | Contrastive training pulls matched pairs together and pushes negatives apart in a shared vector space[1](https://arxiv.org/html/2405.17428v1) |
| Flagship text-embedding score (May 2024) | NV-Embed: 69.32 on MTEB across 56 tasks, vendor-reported No. 1[1](https://arxiv.org/html/2405.17428v1) |
| Typical embedding dimension | 2,048 (jina-embeddings-v4 single-vector; NVIDIA NeMo Retriever per image)[4](https://aclanthology.org/anthology-files/pdf/mrl/2025.mrl-main.36.pdf) • [3](https://developer.nvidia.com/blog/best-in-class-multimodal-rag-how-the-llama-3-2-nemo-retriever-embedding-model-boosts-pipeline-accuracy/) |
| Matryoshka truncation | jina-embeddings-v4 single vectors truncate to as few as 128 dimensions with minimal loss of precision[4](https://aclanthology.org/anthology-files/pdf/mrl/2025.mrl-main.36.pdf) |
| Multimodal coverage | Gemini Embedding 2 embeds video, audio, image, and text in one representation space (2026)[2](https://arxiv.org/html/2605.27295) |
| Score spread among leaders | About 2.7 MTEB points across successive state-of-the-art models (66.63 to 69.32)[1](https://arxiv.org/html/2405.17428v1) |
| Evidence caveat | Headline scores in this article are vendor-reported; no independent reruns appear in the cited sources |
## How the mechanism works: contrastive training
Contrastive training is what makes two related texts' vectors close. The model encodes a query and a set of candidate passages, then a contrastive loss rewards high similarity for the matched pair and low similarity for negatives. In practice the negatives come cheaply from other examples in the same training batch (in-batch negatives), supplemented by curated hard-negative examples that are deliberately similar to the query.[1](https://arxiv.org/html/2405.17428v1)
NV-Embed (NVIDIA, May 2024) illustrates the current recipe. It starts from the pretrained Mistral-7B decoder LLM and applies a two-stage contrastive instruction-tuning method: stage one trains with instructions on retrieval datasets using in-batch negatives and curated hard negatives; stage two blends in curated non-retrieval datasets and disables in-batch negatives, because those samples can mislead on non-retrieval tasks such as clustering or classification.[1](https://arxiv.org/html/2405.17428v1) Two architectural changes mattered in its ablation: removing the LLM's causal attention mask during contrastive training, and replacing mean pooling with a latent-attention pooling layer, which together raised nDCG@10 on the 15 BEIR retrieval tasks from 58.71 to 59.36.[1](https://arxiv.org/html/2405.17428v1)
The backbone has shifted over time from encoder-only models such as BERT and RoBERTa to decoder-only LLM backbones. Along the way, the BGE series and E5 established instruction-tuned representations that unify downstream tasks (semantic search, clustering, classification) into a single model via task-specific prefixes, Gecko showed lightweight retrievers via two-step distillation from LLM teachers, and [NV-Embed](https://www.edgepedia.org/nv-embed) and [Gemini Embedding](https://www.edgepedia.org/gemini-embedding) pushed instruction-tuned contrastive learning further; the Gemini Embedding 2 paper's related-work account credits both with strong MMTEB results using instruction-tuned contrastive learning and synthetic data, though the NV-Embed paper itself states its training data includes no synthetic data from proprietary models.[2](https://arxiv.org/html/2605.27295) • [1](https://arxiv.org/html/2405.17428v1) Instruction prefixes are the practical consequence: the same model serves different tasks because the query or document carries a short task label.
## By the numbers
The most-cited yardstick is MTEB, the Massive Text Embedding Benchmark, which aggregates dozens of embedding tasks including retrieval drawn from BEIR. On MTEB's 56 tasks, the vendor-reported progression among leading models in 2024 was E5-mistral-7b-instruct at 66.63, SFR-[Embedding](https://www.edgepedia.org/embedding) at 67.56, Voyage-large-2-instruct at 68.28, and NV-Embed at 69.32, ranked No. 1 as of May 22, 2024.[1](https://arxiv.org/html/2405.17428v1) NV-Embed scored 59.35 on the 15 MTEB retrieval tasks drawn from BEIR.[1](https://arxiv.org/html/2405.17428v1)
Google's Gemini Embedding 2 (2026) reports 69.9 on MTEB multilingual and 84.0 on MTEB Code, plus 62.9 R@1 on MSCOCO image-text retrieval, and 68.8 NDCG@10 on Vatex video retrieval, claiming to surpass specialized models on these benchmarks.[2](https://arxiv.org/html/2605.27295) Every one of these figures is vendor-reported. The sources available for this article contain no independent leaderboard reruns, so the numbers should be read as each lab's account of its own model rather than as replicated measurements.
## Multimodal and dual-mode embeddings
Text-only encoders are giving way to models that embed several modalities in one space. Gemini Embedding 2 is a native multimodal embedding model: video, audio, image, and text all map into a unified representation space, trained with large-scale contrastive learning in a multi-task, multi-stage setup, and Google reports robust zero-shot performance across domains from astronomy and bioscience to fine arts and the culinary arts, positioned for RAG, recommendation, and search.[2](https://arxiv.org/html/2605.27295)
jina-embeddings-v4 (2025) takes a dual-output approach: it processes text and images and produces both single-vector embeddings (mean-pooled to 2,048 dimensions) and ColBERT-style multi-vector embeddings, with LoRA adapters specialized for retrieval, [semantic similarity](https://www.edgepedia.org/semantic-similarity), and code tasks. Its weights are initialized from Qwen2.5-VL-3B-Instruct, and only the LoRA adapters are trained, with the base model and projection layer remaining as initialized.[4](https://aclanthology.org/anthology-files/pdf/mrl/2025.mrl-main.36.pdf)
For image-heavy RAG, NVIDIA's Llama 3.2 NeMo Retriever Multimodal Embedding 1B model (2024), based on Eagle 2's tiled mixture-of-vision-encoders approach, generates 2,048-dimensional embeddings for each image document and query input.[3](https://developer.nvidia.com/blog/best-in-class-multimodal-rag-how-the-llama-3-2-nemo-retriever-embedding-model-boosts-pipeline-accuracy/)
## Recent developments
Three shifts stand out in the 2024 to 2026 record. First, LLM-backbone embeddings took over the top of MTEB: NV-Embed's May 2024 result of 69.32 came from a decoder-only Mistral-7B start rather than a BERT-style encoder.[1](https://arxiv.org/html/2405.17428v1) Second, embedding models became natively multimodal rather than text-first with bolted-on vision: Gemini Embedding 2 (2026) embeds four modalities in one space,[2](https://arxiv.org/html/2605.27295) and jina-embeddings-v4 (2025) handles text and images with both single- and multi-vector outputs.[4](https://aclanthology.org/anthology-files/pdf/mrl/2025.mrl-main.36.pdf) Third, Matryoshka Representation Learning (Embeddings truncatable to smaller sizes with little accuracy loss) became a shipped feature rather than a research idea: jina-embeddings-v4's single vectors can be truncated to as few as 128 dimensions with minimal loss of precision, letting operators trade vector size against accuracy at query time.[4](https://aclanthology.org/anthology-files/pdf/mrl/2025.mrl-main.36.pdf) Specialization also continued, with code-focused models such as voyage-code and long-document-focused ones such as voyage-3 retaining leads over generalists on their home turf.[4](https://aclanthology.org/anthology-files/pdf/mrl/2025.mrl-main.36.pdf)
## References
1. NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models, https://arxiv.org/html/2405.17428v1
2. Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini, https://arxiv.org/html/2605.27295
3. Best-in-Class Multimodal RAG: Llama 3.2 NeMo Retriever Embedding Model, https://developer.nvidia.com/blog/best-in-class-multimodal-rag-how-the-llama-3-2-nemo-retriever-embedding-model-boosts-pipeline-accuracy/
4. jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrieval, https://aclanthology.org/anthology-files/pdf/mrl/2025.mrl-main.36.pdf
---
*Topic: Encyclopedia › Technology and the built world › Computing and digital systems › Modern AI: foundation models, generative AI, and the AI industry › Foundation-model methods and training › Multimodal, embodied, and world-model methods*
*Initially written Sep 17, 2026 · Edited: Oct 11, 2026; Oct 15, 2026*
*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*
License: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license