# EMMA (Waymo end-to-end driving model)
EMMA (End-to-End Multimodal Model for Autonomous Driving) is an experimental driving model from Waymo, released as arXiv preprint 2410.23262 on 30 October 2024, that runs a single multimodal large language model, Google's Gemini, directly from camera images to driving outputs.[1](https://arxiv.org/html/2410.23262v1) EMMA recasts driving tasks as visual question answering: trajectories, detected objects, and road graph elements are all expressed as natural-language text in one shared representation.[1](https://arxiv.org/html/2410.23262v1) The model was published as a paper only, with no code, weights, or data released.[2](https://openaccess.thecvf.com/content/WACV2025W/LLVMAD/papers/Xing_OpenEMMA_Open-Source_Multimodal_Model_for_End-to-End_Autonomous_Driving_WACVW_2025_paper.pdf) • [3](https://arxiv.org/html/2505.00284)
| Key fact | Detail |
|---|---|
| What it is | Waymo's end-to-end multimodal driving model built on Gemini, with no additional specialized components[1](https://arxiv.org/html/2410.23262v1) |
| Release | arXiv preprint 2410.23262, 30 October 2024; paper only, no code or weights[1](https://arxiv.org/html/2410.23262v1) • [2](https://openaccess.thecvf.com/content/WACV2025W/LLVMAD/papers/Xing_OpenEMMA_Open-Source_Multimodal_Model_for_End-to-End_Autonomous_Driving_WACVW_2025_paper.pdf) |
| Inputs and supervision | Surround-view cameras only; no HD map beyond routing from a navigation system; self-supervised from future ego locations[1](https://arxiv.org/html/2410.23262v1) |
| Best vendor-reported planning result | nuScenes average L2 of 0.32 m (EMMA+ 0.29 m), 17.1% better than self-supervised BEV-Planner[1](https://arxiv.org/html/2410.23262v1) |
| Independent replication | OpenEMMA (December 2024, WACV 2025 workshop) with open VLM backbones achieved far weaker results; no independent confirmation of EMMA's numbers exists[2](https://openaccess.thecvf.com/content/WACV2025W/LLVMAD/papers/Xing_OpenEMMA_Open-Source_Multimodal_Model_for_End-to-End_Autonomous_Driving_WACVW_2025_paper.pdf) |
| Stated deployment obstacles | No camera-LiDAR/radar fusion, costly sensor simulation for closed-loop testing, high compute, few input frames[1](https://arxiv.org/html/2410.23262v1) |
| Deployment status | Not established by available sources[1](https://arxiv.org/html/2410.23262v1) |
## What EMMA is
EMMA applies a language-model backbone to driving policy. The preprint describes it as built on top of Gemini "without additional specialized components": the same multimodal model processes camera images and produces text that encodes the planned future trajectory of the ego vehicle, bounding boxes for objects, and road graph elements.[1](https://arxiv.org/html/2410.23262v1) Waymo's research page summarizes the claimed result as state-of-the-art motion planning on nuScenes and competitive results on the Waymo Open Motion Dataset and camera-primary 3D detection on the [Waymo Open Dataset](https://www.edgepedia.org/waymo-open-dataset).[4](https://waymo.com/research/emma/) Independent papers characterize EMMA as an example of the trend of integrating perception, decision-making, and navigation in a single Gemini-based model.[2](https://openaccess.thecvf.com/content/WACV2025W/LLVMAD/papers/Xing_OpenEMMA_Open-Source_Multimodal_Model_for_End-to-End_Autonomous_Driving_WACVW_2025_paper.pdf)
## Architecture and training as published
Three design choices define the published model. Camera-only: surround-view cameras are the only sensor input, with no LiDAR or radar. HD-map-free: no HD map is used beyond high-level routing from a navigation system such as [Google Maps](https://www.edgepedia.org/google-maps). Self-supervised: for planning, the only supervision is the future locations of the ego vehicle, with no dedicated human labels.[1](https://arxiv.org/html/2410.23262v1)
Training details reported in the preprint: co-training EMMA jointly on planner trajectories, object detection, and road graph tasks improved performance across all three domains, and adding chain-of-thought reasoning plus more internal training data further improved motion-planning quality. A variant called EMMA+ was pre-trained on Waymo's internal mega-scale dataset before fine-tuning, improving Waymo Open Motion Dataset scores over the Gemini-initialized version.[1](https://arxiv.org/html/2410.23262v1) An independent May 2025 paper, LightEMMA, describes EMMA as fine-tuned from Google's pretrained Gemini on Waymo's open and internal datasets.[3](https://arxiv.org/html/2505.00284)
## Benchmark results: vendor versus independent
All headline numbers below are vendor-reported, from Waymo's own preprint; no independent evaluation has reproduced them.
On the Waymo Open Motion Dataset planning benchmark, EMMA+ achieved average displacement error (ADE, the mean distance between predicted and actual future positions) of 0.030 m at 1 s, 0.225 m at 3 s, 0.610 m at 5 s, and 1.553 m at 8 s. This beat MotionLM (0.045/0.251/0.694/1.766 m) and an enhanced internal Wayformer (0.044/0.250/0.640 m) at 1 s to 5 s, but trailed Wayformer at 8 s (1.517 m), which Waymo attributes to higher camera depth error at long range compared with Wayformer's use of LiDAR, radar, and cameras.[1](https://arxiv.org/html/2410.23262v1)
On nuScenes planning, self-supervised EMMA reached average L2 of 0.32 m (EMMA+ 0.29 m), which Waymo reports as 17.1% better than self-supervised BEV-Planner (0.35 m) and 6.4% better than supervised DriveVLM-Dual (0.31 m).[1](https://arxiv.org/html/2410.23262v1) On Waymo Open Dataset camera-primary 3D detection, EMMA+ showed a 16.3% relative improvement in vehicle precision at equal recall, or a 5.5% recall improvement, versus state-of-the-art methods.[1](https://arxiv.org/html/2410.23262v1)
Independent results tell a different story. OpenEMMA, a replication by the TACO group released in December 2024 and peer-reviewed at the WACV 2025 LLVMAD workshop, substituted open VLM backbones (GPT-4, LLaVA-1.6-Mistral-7B, Llama3.2-11B-Vision-Instruct) for the proprietary [Gemini](https://www.edgepedia.org/gemini-model-family). Its authors report only a zero-shot baseline with LLaVA-1.6-Mistral-7B and a modest enhancement with Llama3.2-11B-Vision-Instruct in L2 norm and failure rate, falling well short of EMMA's reported numbers.[2](https://openaccess.thecvf.com/content/WACV2025W/LLVMAD/papers/Xing_OpenEMMA_Open-Source_Multimodal_Model_for_End-to-End_Autonomous_Driving_WACVW_2025_paper.pdf) LightEMMA's authors judged OpenEMMA itself unsuitable for practical use, citing consistently high prediction errors, crashes from insufficient error handling, and an inflexible codebase.[3](https://arxiv.org/html/2505.00284)
## Limitations, reception and the open-loop debate
Waymo's own paper lists the obstacles to real-world deployment: limited 3D spatial reasoning because EMMA cannot fuse camera inputs with LiDAR or radar; the need for realistic and computationally expensive sensor simulation to power closed-loop evaluation, which the paper says costs several times more than behavior simulators; and increased computational requirements relative to conventional models. The model can also process only a small number of image frames.[1](https://arxiv.org/html/2410.23262v1)
The paper also concedes a methodological point that became the core of outside criticism: "It is widely recognized that open-loop evaluation may not be strongly correlated with closed-loop performance."[1](https://arxiv.org/html/2410.23262v1) Open-loop evaluation involves replaying logs, which, per a 2025 writeup of a talk by Waymo researcher Jyh Jing Hwang, Waymo considers insufficient for understanding EMMA's behavior; Waymo instead uses generative video models (Drive&Gen) for closed-loop simulation, and controlled testing there showed the camera-only planner's performance degrades slightly in adverse weather and at night.[5](https://tokenless.tech/posts/2025/waymo-research-emma-end-to-end-autonomous-driving/)
## Availability, licensing and reproducibility
EMMA was released as a paper only. No code, model weights, or training data were published, and both independent papers describe it as proprietary and not open-source; LightEMMA's authors say this limits accessibility and impedes research collaboration.[2](https://openaccess.thecvf.com/content/WACV2025W/LLVMAD/papers/Xing_OpenEMMA_Open-Source_Multimodal_Model_for_End-to-End_Autonomous_Driving_WACVW_2025_paper.pdf) • [3](https://arxiv.org/html/2505.00284) The proprietary status prompted replication efforts: OpenEMMA fully released its codebase, datasets, and model weights on GitHub,[2](https://openaccess.thecvf.com/content/WACV2025W/LLVMAD/papers/Xing_OpenEMMA_Open-Source_Multimodal_Model_for_End-to-End_Autonomous_Driving_WACVW_2025_paper.pdf) and LightEMMA followed in May 2025, positioning itself as the first systematic evaluation of VLMs in autonomous driving across inference time, computational efficiency, prediction accuracy, and common failure cases.[3](https://arxiv.org/html/2505.00284)
## References
1. [EMMA: End-to-End Multimodal Model for Autonomous Driving (arXiv 2410.23262)](https://arxiv.org/html/2410.23262v1)
2. [OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Driving (WACV 2025 LLVMAD workshop)](https://openaccess.thecvf.com/content/WACV2025W/LLVMAD/papers/Xing_OpenEMMA_Open-Source_Multimodal_Model_for_End-to-End_Autonomous_Driving_WACVW_2025_paper.pdf)
3. [LightEMMA: Lightweight End-to-End Multimodal Model for Autonomous Driving (arXiv 2505.00284)](https://arxiv.org/html/2505.00284)
4. [EMMA (Waymo research page)](https://waymo.com/research/emma/)
5. [Waymo's EMMA: Teaching Cars to Think (Tokenless, 2025)](https://tokenless.tech/posts/2025/waymo-research-emma-end-to-end-autonomous-driving/)
---
*Topic: Encyclopedia › Technology and the built world › Computing and digital systems › Modern AI: foundation models, generative AI, and the AI industry › Model families and named models › Multimodal, vision, and world models*
*Initially written Sep 17, 2026 · Edited: Oct 11, 2026; Oct 15, 2026*
*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*
License: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license