# Vision–language–action model
A vision–language–action (VLA) model is a robotic AI system that takes visual observations and natural language instructions as required inputs and produces robot actions by directly generating control commands, rather than merely selecting from a set of pre-trained skills.[1](https://doi.org/10.1109/ACCESS.2025.3609980) When the architecture incorporates a pre-trained large language model or vision-language model, it may also be termed a large vision-language-action model (LVLA).[2](https://www.mdpi.com/2218-6581/15/8/160) VLA models unify perception, reasoning, and control in a single multimodal framework, replacing the modular perception-reasoning-control pipelines used in earlier robotics systems.[3](https://www.elspub.com/doi/10.55092/rl20260010)
| Key fact | Detail |
|---|---|
| Definition | Maps visual observations and language instructions directly to robot control commands; skill-selection-only systems are excluded from the definition[1](https://doi.org/10.1109/ACCESS.2025.3609980) |
| First prominent example | RT-2 (Google DeepMind, 2023), built on PaLM-E and PaLI-X vision-language models combined with robotic data[4](https://deepmind.google/blog/rt-2-new-model-translates-vision-and-language-into-action/) |
| Flagship open model | OpenVLA, 7B parameters, trained on 970k real-world robot demonstrations[5](https://arxiv.org/abs/2406.09246) |
| Key dataset | Open X-Embodiment repository, curated from 21 institutions across 22 robotic embodiments[6](https://robotics-transformer-x.github.io/paper.pdf) |
| Reported efficiency | OpenVLA outperformed RT-2-X (55B) by 16.5% absolute task success rate across 29 tasks with 7x fewer parameters[5](https://arxiv.org/abs/2406.09246) |
| Main constraints | Scarce tri-modal (vision, language, action) data, embodiment mismatch, and computational cost[1](https://doi.org/10.1109/ACCESS.2025.3609980) |
## Architecture
Modern VLA systems decompose into three components: multimodal perception encoders, cross-modal fusion mechanisms, and action decoders.[3](https://www.elspub.com/doi/10.55092/rl20260010) The typical recipe starts from a pre-trained vision-language model (VLM) and adapts it to the robotics domain so that language and visual understanding are grounded in action.[1](https://doi.org/10.1109/ACCESS.2025.3609980) Action representation in end-to-end VLA models falls into several primary approaches, and design choices vary across model architecture, training data, action representation, and robotic embodiment.[1](https://doi.org/10.1109/ACCESS.2025.3609980) • [2](https://www.mdpi.com/2218-6581/15/8/160)
The value of this design comes from transfer.
## Origin: RT-2 and the RT series
Robotics [Transformer](https://www.edgepedia.org/transformer) 2 (RT-2), announced by [Google DeepMind](https://www.edgepedia.org/google-deepmind) in 2023, is described as a first-of-its-kind VLA model and established the category: it showed that vision-language models can be transformed into models that directly control a robot by combining VLM pre-training with robotic data.[7](https://blog.google/innovation-and-ai/products/google-deepmind-rt2-robotics-vla-model/) • [4](https://deepmind.google/blog/rt-2-new-model-translates-vision-and-language-into-action/) Two instantiations were built, based on PaLM-E and PaLI-X, and the resulting policies showed significantly better generalization and emergent capabilities inherited from web-scale pre-training.[4](https://deepmind.google/blog/rt-2-new-model-translates-vision-and-language-into-action/) The RT-2 paper introduced the term "vision-language-action models" for this category.[8](https://robotics-transformer2.github.io/assets/rt2.pdf)
## Open models and large-scale data
Widespread adoption of VLAs was initially held back because most models were closed and methods for efficient fine-tuning were unexplored.[5](https://arxiv.org/abs/2406.09246) OpenVLA, released in June 2024, addressed both gaps: it is a 7B-parameter open-source model built on a [Llama 2](https://www.edgepedia.org/llama-2) language model with a visual encoder fusing pretrained DINOv2 and SigLIP features, trained on 970k real-world robot demonstrations.[5](https://arxiv.org/abs/2406.09246) It outperformed the closed RT-2-X (55B) by 16.5% in absolute task success rate across 29 tasks and multiple robot embodiments despite having 7 times fewer parameters, beat the from-scratch [Diffusion Policy](https://www.edgepedia.org/diffusion-policy) by 20.4%, and can be fine-tuned on consumer GPUs with low-rank adaptation and served via quantization without loss of success rate.[5](https://arxiv.org/abs/2406.09246)
Training data at this scale comes from the [Open X-Embodiment](https://www.edgepedia.org/open-x-embodiment) (OXE) repository, an open, large-scale robot learning dataset curated from 21 institutions worldwide and covering 22 different robotic embodiments, provided in standardized formats together with the [RT-X](https://www.edgepedia.org/rt-x) models.[6](https://robotics-transformer-x.github.io/paper.pdf) Compact alternatives trade scale for efficiency: TinyVLA eliminates the pre-training stage and outperforms OpenVLA in inference speed and data efficiency while delivering comparable or superior task performance.[9](https://arxiv.org/abs/2409.12514)
## The Gemini Robotics family
Google DeepMind's Gemini Robotics line illustrates the current generation of VLAs. [Gemini Robotics](https://www.edgepedia.org/gemini-robotics), introduced on March 12, 2025, is built on [Gemini 2](https://www.edgepedia.org/gemini-2).0 with physical actions added as a new output modality for directly controlling robots; it was released alongside Gemini Robotics-ER, a model with advanced spatial understanding that lets roboticists run their own programs using embodied reasoning.[10](https://deepmind.google/blog/gemini-robotics-brings-ai-into-the-physical-world/) In June 2025 the company released Gemini Robotics On-Device, a VLA optimized to run locally on the robot, which is useful for latency-sensitive applications and environments with intermittent connectivity; it adapts to new tasks with as few as 50 to 100 demonstrations and was trained on ALOHA robots and adapted to the bi-arm Franka FR3 and the Apollo humanoid.[11](https://deepmind.google/blog/gemini-robotics-on-device-brings-ai-to-local-robotic-devices/) Gemini Robotics 1.5, introduced in September 2025, pairs the VLA with the Gemini Robotics-ER 1.5 embodied reasoning model in an agentic framework: the ER model plans, calls tools such as [Google Search](https://www.edgepedia.org/google-search), and issues natural-language instructions that the VLA executes, and the VLA can generate internal reasoning before acting and transfer learned motions across embodiments without per-robot specialization.[12](https://deepmind.google/blog/gemini-robotics-15-brings-ai-agents-into-the-physical-world/)
Other companies have entered the space. Microsoft announced Rho-alpha, a robotics model derived from its Phi vision-language series that translates natural language commands into robotic control signals for bimanual manipulation and adds tactile sensing alongside vision.[13](https://www.techradar.com/pro/microsoft-unveils-first-robotics-model-targeted-at-boosting-physical-ai-in-a-bid-to-free-robots-from-the-production-line) Alibaba's Tongyi Lab released the Qwen Robot Suite, whose Qwen-RobotManip is a generalizable VLA model complemented by a vision-language navigation model and a video world model.[14](https://www.techradar.com/pro/look-out-nvidia-alibaba-reveals-its-most-powerful-ai-models-for-robots-as-it-looks-to-strike-ahead-in-agentic-race)
## Evaluation and generalization
Systematic evaluation remains a weak point. A benchmarking study profiling GPT-4o, OpenVLA, and JAT across 20 datasets from the Open X-Embodiment collection found significant performance variation across tasks and robot platforms, that all models struggle with complex manipulation requiring multi-step planning, and that performance is notably sensitive to action space characteristics and environmental factors.[15](https://arxiv.org/abs/2411.05821) Surveys of the benchmark landscape report [benchmark saturation](https://www.edgepedia.org/benchmark-saturation), narrow metrics, real-robot fragmentation, and a proxy-validity gap in which simulation scores are read as evidence of real-world readiness without paired physical validation.[16](https://www.preprints.org/manuscript/202606.0425) Newer benchmarks target these gaps: RADAR evaluates VLA generalization under realistic conditions including dynamic object configurations, robot initial states, lighting changes, and sensor noise;[17](https://arxiv.org/abs/2602.10980) VLA-Arena reports that state-of-the-art VLAs show memorization over generalization, superficial visual perception, and neglect of safety constraints;[18](https://vla-arena.github.io/) and the ROEP protocol specifies which deployment-relevant claims a benchmark result actually supports, without certifying open-world deployment safety.[19](https://www.mdpi.com/1424-8220/26/15/4757)
## Challenges
Three constraints recur across surveys. Data scarcity: training requires large-scale data aligning vision, language, and action, and datasets containing all three modalities are limited in scale and diversity, since web-scale vision-language corpora lack action grounding and robot demonstrations have limited linguistic variability.[1](https://doi.org/10.1109/ACCESS.2025.3609980) Embodiment mismatch: robots differ in shape, sensing, and degrees of freedom, making transfer of learned motions between robots difficult, though cross-embodiment datasets and models such as Gemini Robotics 1.5 address this directly.[1](https://doi.org/10.1109/ACCESS.2025.3609980) • [12](https://deepmind.google/blog/gemini-robotics-15-brings-ai-agents-into-the-physical-world/) Computational cost: the computational burden of VLAs limits accessibility and deployability, motivating efficient architectures and distillation methods.[1](https://doi.org/10.1109/ACCESS.2025.3609980)
Inference latency produces a visible behavior: VLA-controlled robots often move in fits and starts because the robot waits for the model to compute the next action set. A system called VLASH plans the next actions while the robot executes current ones; in tests, tasks completed 1.5 to 2 times faster with most or all accuracy retained, and maximum reaction latency fell by up to 11.8 times depending on hardware.[20](https://www.livescience.com/technology/robotics/new-ai-technique-helps-robots-complete-tasks-twice-as-fast-by-letting-them-think-ahead) A literature review also finds that few studies incorporate systematic assessments of operational safety, failure behavior, resilience to unexpected inputs, or security risks, even though these matter for real-world deployment.[2](https://www.mdpi.com/2218-6581/15/8/160)
## References
1. [Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications, IEEE Access (2025)](https://doi.org/10.1109/ACCESS.2025.3609980)
2. [Systematic Literature Review of Vision-Language-Action Models for Generalist Robots, Robotics (MDPI)](https://www.mdpi.com/2218-6581/15/8/160)
3. [The evolution of end-to-end Vision-Language-Action (VLA) architectures in robotics, ELSPublishing](https://www.elspub.com/doi/10.55092/rl20260010)
4. [RT-2: New model translates vision and language into action, Google DeepMind](https://deepmind.google/blog/rt-2-new-model-translates-vision-and-language-into-action/)
5. [OpenVLA: An Open-Source Vision-Language-Action Model, arXiv](https://arxiv.org/abs/2406.09246)
6. [Open X-Embodiment: Robotic Learning Datasets and RT-X Models](https://robotics-transformer-x.github.io/paper.pdf)
7. [What is RT-2? Google DeepMind's vision-language-action model for robotics](https://blog.google/innovation-and-ai/products/google-deepmind-rt2-robotics-vla-model/)
8. [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control (paper page)](https://robotics-transformer2.github.io/assets/rt2.pdf)
9. [TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation, arXiv](https://arxiv.org/abs/2409.12514)
10. [Introducing Gemini Robotics and Gemini Robotics-ER, Google DeepMind](https://deepmind.google/blog/gemini-robotics-brings-ai-into-the-physical-world/)
11. [Gemini Robotics On-Device brings AI to local robotic devices, Google DeepMind](https://deepmind.google/blog/gemini-robotics-on-device-brings-ai-to-local-robotic-devices/)
12. [Gemini Robotics 1.5 brings AI agents into the physical world, Google DeepMind](https://deepmind.google/blog/gemini-robotics-15-brings-ai-agents-into-the-physical-world/)
13. [Microsoft unveils first robotics model targeted at boosting physical AI, TechRadar](https://www.techradar.com/pro/microsoft-unveils-first-robotics-model-targeted-at-boosting-physical-ai-in-a-bid-to-free-robots-from-the-production-line)
14. [Alibaba reveals its most powerful AI models for robots, TechRadar](https://www.techradar.com/pro/look-out-nvidia-alibaba-reveals-its-most-powerful-ai-models-for-robots-as-it-looks-to-strike-ahead-in-agentic-race)
15. [Benchmarking Vision, Language, & Action Models on Robotic Learning Tasks, arXiv](https://arxiv.org/abs/2411.05821)
16. [The Evaluation Bottleneck of Vision-Language-Action Models: An Evaluation-Centric Survey, Preprints.org](https://www.preprints.org/manuscript/202606.0425)
17. [RADAR: Benchmarking Vision-Language-Action Generalization via Real-World Dynamics, arXiv](https://arxiv.org/abs/2602.10980)
18. [VLA-Arena: VLA Benchmark and Leaderboard](https://vla-arena.github.io/)
19. [ROEP: A Robotics-Oriented Evaluation Protocol for Deployment-Facing Vision-Language-Action Manipulation Policies, Sensors (MDPI)](https://www.mdpi.com/1424-8220/26/15/4757)
20. [New AI technique helps robots complete tasks twice as fast by letting them 'think ahead', Live Science](https://www.livescience.com/technology/robotics/new-ai-technique-helps-robots-complete-tasks-twice-as-fast-by-letting-them-think-ahead)
---
*Topic: Encyclopedia › Technology and the built world › Computing and digital systems › Modern AI: foundation models, generative AI, and the AI industry › Foundation-model methods and training › Multimodal, embodied, and world-model methods*
*Initially written Sep 17, 2026 · Reviewed: Sep 30, 2026; Oct 15, 2026 · Edited: Oct 15, 2026 · Last review: Oct 15, 2026*
*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*
License: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license