# VideoWorld
VideoWorld is an auto-regressive video generation model, introduced in January 2025 by ByteDance's Seed team with Beijing Jiaotong University and the [University of Science and Technology of China](https://www.edgepedia.org/university-of-science-and-technology-of-china), that is trained on unlabeled task-execution videos to acquire rules, reasoning, and planning abilities and to produce control actions, without language supervision or reinforcement learning signals.[1](https://openaccess.thecvf.com/content/CVPR2025/html/Ren_VideoWorld_Exploring_Knowledge_Learning_from_Unlabeled_Videos_CVPR_2025_paper.html) • [2](https://seed.bytedance.com/en/blog/seed-research-latest-breakthrough-in-video-generation-learning-to-understand-the-world-through-vision-alone-now-open-source) In practice, "learning purely from video" means the raw task-execution videos collected from the experimental environments are the model's exclusive knowledge source: the model watches Go games or robot manipulation episodes as frames, and the Go moves or robotic actions it must output are inferred from the frames it generates.[3](https://arxiv.org/pdf/2501.09781v2.pdf) A second generation, VideoWorld 2 (CVPR 2026), extends the approach to transferable knowledge for long-horizon real-world tasks learned from raw real-world videos.[4](https://openaccess.thecvf.com/content/CVPR2026/html/Ren_VideoWorld_2_Learning_Transferable_Knowledge_from_Real-world_Videos_CVPR_2026_paper.html)
Published benchmark results for both generations are reported in the papers by the model's authors.[1](https://openaccess.thecvf.com/content/CVPR2025/html/Ren_VideoWorld_Exploring_Knowledge_Learning_from_Unlabeled_Videos_CVPR_2025_paper.html) • [4](https://openaccess.thecvf.com/content/CVPR2026/html/Ren_VideoWorld_2_Learning_Transferable_Knowledge_from_Real-world_Videos_CVPR_2026_paper.html)
| Fact | Value |
|---|---|
| First generation | VideoWorld, CVPR 2025, preprint and repository January 2025[1](https://openaccess.thecvf.com/content/CVPR2025/html/Ren_VideoWorld_Exploring_Knowledge_Learning_from_Unlabeled_Videos_CVPR_2025_paper.html) • [5](https://github.com/ByteDance-Seed/VideoWorld/) |
| Second generation | VideoWorld 2, CVPR 2026, code and dataset released February 2026[6](https://github.com/ByteDance-Seed/VideoWorld/blob/main/VideoWorld2/README.md) |
| Parameter counts | 300M (headline Go result); a 50M variant is also reported[1](https://openaccess.thecvf.com/content/CVPR2025/html/Ren_VideoWorld_Exploring_Knowledge_Learning_from_Unlabeled_Videos_CVPR_2025_paper.html) • [3](https://arxiv.org/pdf/2501.09781v2.pdf) |
| License | Apache License 2.0, code, data, and models open-sourced[5](https://github.com/ByteDance-Seed/VideoWorld/) |
| Headline results (vendor) | 5-dan on Video-GoBench at 300M; near-oracle CALVIN/RLBench; up to 70% task-success gain on Video-Craft[1](https://openaccess.thecvf.com/content/CVPR2025/html/Ren_VideoWorld_Exploring_Knowledge_Learning_from_Unlabeled_Videos_CVPR_2025_paper.html) • [4](https://openaccess.thecvf.com/content/CVPR2026/html/Ren_VideoWorld_2_Learning_Transferable_Knowledge_from_Real-world_Videos_CVPR_2026_paper.html) |
| Training compute (v1) | About 4 days (Go) and 2 days (CALVIN) on 8 A100 GPUs[3](https://arxiv.org/pdf/2501.09781v2.pdf) |
| Adoption signal | 791 stars, 40 forks, 13 open issues on GitHub as of September 2026[5](https://github.com/ByteDance-Seed/VideoWorld/) |
## How it works
The first-generation model converts video frames into discrete tokens with a VQ-VAE and trains an auto-regressive transformer on those tokens under the next-token (next-frame) prediction paradigm, in the manner of large language models.[3](https://arxiv.org/pdf/2501.09781v2.pdf) The encoder-decoder follows a MAGVITv2-style causal design, with temporal downsampling intentionally omitted to preserve detail in each frame.[2](https://seed.bytedance.com/en/blog/seed-research-latest-breakthrough-in-video-generation-learning-to-understand-the-world-through-vision-alone-now-open-source)
The core component is a Latent Dynamics Model (LDM). Its encoder extracts features causally, and attention blocks capture frame-to-frame change into a continuous latent that is quantized by finite scalar quantization (FSQ). The authors describe the quantizer as an [information bottleneck](https://www.edgepedia.org/information-bottleneck) that prevents the LDM from merely memorizing the original content of subsequent frames; the paper's second stated finding is that representing visual change is crucial for knowledge acquisition.[2](https://seed.bytedance.com/en/blog/seed-research-latest-breakthrough-in-video-generation-learning-to-understand-the-world-through-vision-alone-now-open-source) • [1](https://openaccess.thecvf.com/content/CVPR2025/html/Ren_VideoWorld_Exploring_Knowledge_Learning_from_Unlabeled_Videos_CVPR_2025_paper.html) The default vocabulary size for both the LDM and transformer encoders is 64,000, corresponding to FSQ levels [8, 8, 8, 5, 5, 5].[3](https://arxiv.org/pdf/2501.09781v2.pdf)
VideoWorld 2 changes the architecture to address long-horizon real-world tasks. Its dynamics-enhanced Latent Dynamics Model (dLDM) decouples action dynamics from visual appearance: a pretrained video diffusion model handles visual appearance modeling, while latent codes are modeled autoregressively to learn task policies and support long-horizon reasoning.[4](https://openaccess.thecvf.com/content/CVPR2026/html/Ren_VideoWorld_2_Learning_Transferable_Knowledge_from_Real-world_Videos_CVPR_2026_paper.html) The appearance model builds on NVIDIA's Cosmos-Predict2-2B-Video2World pretrained weights, which users must download separately.[6](https://github.com/ByteDance-Seed/VideoWorld/blob/main/VideoWorld2/README.md)
## Versions and release timeline
The repository was created on 15 January 2025 and its last push was on 25 February 2026.[5](https://github.com/ByteDance-Seed/VideoWorld/) The repo describes VideoWorld (CVPR 2025) as the first generation, using the Latent Dynamics Model, and VideoWorld 2 (CVPR 2026) as the second generation, focused on transferable knowledge using a disentangled Latent Dynamics Model.[5](https://github.com/ByteDance-Seed/VideoWorld/) VideoWorld 2 was accepted by CVPR 2026, and its code and dataset were released in February 2026.[6](https://github.com/ByteDance-Seed/VideoWorld/blob/main/VideoWorld2/README.md)
## By the numbers
All results in this section are reported by the model's authors.
**Go.** VideoWorld reaches a 5-dan professional level in Video-GoBench with a 300-million-parameter model, without relying on the search algorithms or reward mechanisms typical in reinforcement learning.[1](https://openaccess.thecvf.com/content/CVPR2025/html/Ren_VideoWorld_Exploring_Knowledge_Learning_from_Unlabeled_Videos_CVPR_2025_paper.html)
**Robotic control.** In robotic tasks, VideoWorld learns diverse control operations and generalizes across environments, approaching the performance of oracle models in [CALVIN](https://www.edgepedia.org/calvin) and [RLBench](https://www.edgepedia.org/rlbench).[1](https://openaccess.thecvf.com/content/CVPR2025/html/Ren_VideoWorld_Exploring_Knowledge_Learning_from_Unlabeled_Videos_CVPR_2025_paper.html) CALVIN training and testing used the ABCD→D split, with RLBench using 20,000 trajectories.[3](https://arxiv.org/pdf/2501.09781v2.pdf)
**Real-world tasks.** VideoWorld 2 achieves up to a 70% improvement in task success rate on real-world handcraft making tasks, where the authors report that prior video generation and latent-dynamics models struggle, and produces coherent long execution videos.[4](https://openaccess.thecvf.com/content/CVPR2026/html/Ren_VideoWorld_2_Learning_Transferable_Knowledge_from_Real-world_Videos_CVPR_2026_paper.html) Pretraining on the Open-X dataset substantially improves task performance on CALVIN, which the authors present as cross-domain generalization.[4](https://openaccess.thecvf.com/content/CVPR2026/html/Ren_VideoWorld_2_Learning_Transferable_Knowledge_from_Real-world_Videos_CVPR_2026_paper.html)
**Training cost.** Training used AdamW with a learning rate of 0.0003 and no weight decay, batch size 256 for Go and 32 for CALVIN, requiring approximately 4 and 2 days on 8 A100 GPUs respectively.[3](https://arxiv.org/pdf/2501.09781v2.pdf)
## Data, availability and licensing
The first-generation model was trained on raw task-execution videos collected from the experimental environments (Go games and robotic simulation), which the paper calls the exclusive knowledge source.[3](https://arxiv.org/pdf/2501.09781v2.pdf) VideoWorld 2 was trained on the Video-CraftBench and OpenX datasets; Video-CraftBench data, including paper and block clips, original MP4s, and keyframe annotations, is downloadable from the repository.[6](https://github.com/ByteDance-Seed/VideoWorld/blob/main/VideoWorld2/README.md) The Video-Craft benchmark was established to evaluate models on real-world handicraft making and robotic manipulation.[4](https://openaccess.thecvf.com/content/CVPR2026/html/Ren_VideoWorld_2_Learning_Transferable_Knowledge_from_Real-world_Videos_CVPR_2026_paper.html)
The repository is licensed under the [Apache License](https://www.edgepedia.org/apache-license) 2.0 and is written almost entirely in Python (99.5%).[5](https://github.com/ByteDance-Seed/VideoWorld/) Both papers state that all code, data, and models are open-sourced.[1](https://openaccess.thecvf.com/content/CVPR2025/html/Ren_VideoWorld_Exploring_Knowledge_Learning_from_Unlabeled_Videos_CVPR_2025_paper.html) • [4](https://openaccess.thecvf.com/content/CVPR2026/html/Ren_VideoWorld_2_Learning_Transferable_Knowledge_from_Real-world_Videos_CVPR_2026_paper.html) Reproducing VideoWorld 2 additionally requires downloading the Cosmos-Predict2-2B-Video2World pretrained weights beforehand.[6](https://github.com/ByteDance-Seed/VideoWorld/blob/main/VideoWorld2/README.md)
## References
1. [VideoWorld: Exploring Knowledge Learning from Unlabeled Videos (CVPR 2025)](https://openaccess.thecvf.com/content/CVPR2025/html/Ren_VideoWorld_Exploring_Knowledge_Learning_from_Unlabeled_Videos_CVPR_2025_paper.html)
2. [Seed News: VideoWorld open-source announcement](https://seed.bytedance.com/en/blog/seed-research-latest-breakthrough-in-video-generation-learning-to-understand-the-world-through-vision-alone-now-open-source)
3. [VideoWorld: Exploring Knowledge Learning from Unlabeled Videos (arXiv 2501.09781)](https://arxiv.org/pdf/2501.09781v2.pdf)
4. [VideoWorld 2: Learning Transferable Knowledge from Real-world Videos (CVPR 2026)](https://openaccess.thecvf.com/content/CVPR2026/html/Ren_VideoWorld_2_Learning_Transferable_Knowledge_from_Real-world_Videos_CVPR_2026_paper.html)
5. [ByteDance-Seed/VideoWorld GitHub repository](https://github.com/ByteDance-Seed/VideoWorld/)
6. [VideoWorld2 README](https://github.com/ByteDance-Seed/VideoWorld/blob/main/VideoWorld2/README.md)
---
*Topic: Encyclopedia › Technology and the built world › Computing and digital systems › Modern AI: foundation models, generative AI, and the AI industry › Model families and named models › Multimodal, vision, and world models*
*Initially written Sep 17, 2026 · Edited: Oct 11, 2026; Oct 15, 2026*
*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*
License: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license