# veRL
veRL (styled "verl") is an open-source reinforcement learning post-training framework for large language models, originating from ByteDance's Seed MLSys team and first published as a repository on 31 October 2024 under the [Apache License](https://www.edgepedia.org/apache-license) 2.0.[1](https://github.com/verl-project/verl?tab=readme-ov-file) It is the open-source implementation of HybridFlow, a framework design described in a 2024 paper by [ByteDance](https://www.edgepedia.org/bytedance) and [University of Hong Kong](https://www.edgepedia.org/university-of-hong-kong) researchers, and adoption proxies such as GitHub stars and PyPI downloads indicate substantial use for RLHF post-training.[2](https://verl.readthedocs.io/en/latest/) • [1](https://github.com/verl-project/verl?tab=readme-ov-file)
| Key fact | Detail |
|---|---|
| Origin | ByteDance Seed MLSys team; repository created 31 October 2024[1](https://github.com/verl-project/verl?tab=readme-ov-file) • [3](https://pypi.org/project/verl/0.7.1/) |
| License | Apache-2.0; Python >=3.10[3](https://pypi.org/project/verl/0.7.1/) |
| Design basis | HybridFlow paper (arXiv:2409.19256, 2024, ByteDance + HKU)[1](https://github.com/verl-project/verl?tab=readme-ov-file) |
| Backends | FSDP, FSDP2, Megatron-LM for training; vLLM, SGLang, HF Transformers for rollout[1](https://github.com/verl-project/verl?tab=readme-ov-file) |
| Algorithms | PPO, GRPO, GSPO, ReMax, REINFORCE++, RLOO, PRIME, DAPO, DrGRPO, KL_Cov, Clip_Cov[1](https://github.com/verl-project/verl?tab=readme-ov-file) |
| Scale ceiling | 671B-parameter models on hundreds of GPUs with expert parallelism (vendor-reported)[1](https://github.com/verl-project/verl?tab=readme-ov-file) |
| Adoption (Sept 2026) | ~23,000 GitHub stars, ~108,000 monthly PyPI downloads[1](https://github.com/verl-project/verl?tab=readme-ov-file) • [3](https://pypi.org/project/verl/0.7.1/) |
## Origins and release history
The framework's design was published in "HybridFlow: A Flexible and Efficient RLHF Framework" (arXiv:2409.19256, 2024), authored by Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu, a collaboration between ByteDance and the University of Hong Kong.[1](https://github.com/verl-project/verl?tab=readme-ov-file) The public repository followed on 31 October 2024, and PyPI metadata lists the package author as "Bytedance - Seed - MLSys".[1](https://github.com/verl-project/verl?tab=readme-ov-file) • [3](https://pypi.org/project/verl/0.7.1/)
The most fully documented release in the retrieved record is v0.7 (with v0.7.1 on PyPI by September 2026), which reworked the rollout path: the legacy SPMD (single program, multiple data parallel execution style) rollout mode was removed and replaced by a rollout server mode using online serving with dynamic batching, integrating vLLM, SGLang, and [TensorRT-LLM](https://www.edgepedia.org/tensorrt-llm) backends.[4](https://github.com/verl-project/verl/blob/main/docs/blog/v0.7.md) v0.7 also added an AgentLoop abstraction, with SingleTurnAgentLoop and ToolAgentLoop (ReAct-style multi-turn tool invocation) as default implementations for agentic RL, a fully asynchronous mode that decouples the Trainer and Rollouter onto separate nodes with streaming transfer, staleness control, and partial rollout, and a refactor of reward models into server mode supporting colocated and standalone deployments with hybrid generative, discriminative, and rule-based reward managers.[4](https://github.com/verl-project/verl/blob/main/docs/blog/v0.7.md) Detailed per-version dates and features for releases before v0.7 are not settled by the sources retrieved here.
## How it works
verl models RL algorithms such as PPO, GRPO, and [DAPO](https://www.edgepedia.org/dapo) as a multi-stage, multi-model, parallelizable dataflow graph under a hybrid-controller architecture. A single controller orchestrates the overall program in MPMD (multiple programs, multiple data) style, while the individual model roles (actor, critic, reward model, rollout generator) run internally as SPMD model engines.[4](https://github.com/verl-project/verl/blob/main/docs/blog/v0.7.md) The project states this design shares principles with asynchronous sharded dataflow systems such as Google Pathways.[4](https://github.com/verl-project/verl/blob/main/docs/blog/v0.7.md)
Complex parallel strategies, including 5D parallelism (data, tensor, context, pipeline, and expert parallel), are encapsulated strictly within the Model Engine, so the controller does not manage per-device placement. GPU sharing among the actor, critic, reward, and rollout roles is handled through Ray placement groups exposed as ResourcePool and WorkerGroup abstractions.[4](https://github.com/verl-project/verl/blob/main/docs/blog/v0.7.md)
For training, verl supports PyTorch FSDP, FSDP2, and [Megatron-LM](https://www.edgepedia.org/megatron-lm); for rollout generation it supports vLLM, SGLang, and HF Transformers.[1](https://github.com/verl-project/verl?tab=readme-ov-file) The v0.7 fully async mode, which separates training and generation onto different nodes and streams samples between them, is described by the project as essential for large runs of 128+ GPUs or long chain-of-thought tasks, where generation latency otherwise bottlenecks throughput.[4](https://github.com/verl-project/verl/blob/main/docs/blog/v0.7.md)
## By the numbers
The vendor reports that verl scales to 671B-parameter models on hundreds of GPUs with expert parallelism, and runs on NVIDIA, AMD, and Ascend hardware.[1](https://github.com/verl-project/verl?tab=readme-ov-file) A third-party write-up in Starlog, repeating vendor-adjacent claims, reports that ByteDance runs production RLHF on trillion-parameter models with verl and cites benchmarks showing 2-3x throughput improvements over naive implementations; no independent benchmark confirming that figure appears in the retrieved record.[5](https://starlog.is/articles/llm-engineering/verl-project-verl/)
Adoption proxies as of the September 2026 retrieval: roughly 23,000 GitHub stars and 4,400 forks (retrievals differ slightly, 22,769-23,025 stars and 4,332-4,425 forks), about 1,100 open issues, and PyPI downloads of 4,226 per day, 26,681 per week, and 108,156 per month.[1](https://github.com/verl-project/verl?tab=readme-ov-file) • [3](https://pypi.org/project/verl/0.7.1/)
## Models and results trained with verl
Starlog reports that teams implemented DAPO (50 points on AIME 2024), [VAPO](https://www.edgepedia.org/vapo) (60.4), and Seed-Thinking-v1.5 (86.7 on benchmarks) on verl by modifying only the algorithm layer; these are vendor-reported results repeated by a third party, with no independent confirmation in the retrieved record.[5](https://starlog.is/articles/llm-engineering/verl-project-verl/) A disagreement exists over which benchmarks the Seed-Thinking-v1.5 figure covers: project release notes (April 2025) reportedly give 86.7 on AIME 2024, 55.0 on [Codeforces](https://www.edgepedia.org/codeforces), and 77.3 on GPQA, while Starlog gives only an unspecified "86.7 on benchmarks"; the discrepancy is unresolved here.
verl v0.7.1 is compatible with [Hugging Face Transformers](https://www.edgepedia.org/hugging-face-transformers) and ModelScope Hub models including Qwen-3, Qwen-2.5, Llama3.1, Gemma2, and DeepSeek-LLM, and the framework supports vision-language and multimodal RL with Qwen2.5-VL and [Kimi-VL](https://www.edgepedia.org/kimi-vl), plus multi-turn tool calling.[3](https://pypi.org/project/verl/0.7.1/) • [1](https://github.com/verl-project/verl?tab=readme-ov-file)
## How it compares with TRL, OpenRLHF and other frameworks
The only comparative guidance in the retrieved record is Starlog's practitioner advice: verl suits production-scale RLHF and GRPO on large models and multi-node clusters, while for initial RLHF experiments with models under 10B parameters or single-node setups it recommends HuggingFace TRL for simplicity or OpenRLHF as a gentler introduction to distributed RLHF.[5](https://starlog.is/articles/llm-engineering/verl-project-verl/) On hardware sizing, Starlog reports example configurations using 64+ H800 GPUs for trillion-parameter models, and that 30B-70B experiments work best with at least 16-32 GPUs; no dollar or GPU-hour cost figures for an R1-style run appear in the sources.[5](https://starlog.is/articles/llm-engineering/verl-project-verl/)
## Adoption and reception
Aggregate signals (stars, forks, PyPI downloads, open issues) indicate substantial adoption, and ByteDance is named as a user of verl.[1](https://github.com/verl-project/verl?tab=readme-ov-file) • [3](https://pypi.org/project/verl/0.7.1/) Reception in the record is mixed in a specific way: the framework is valued for production-scale RL, and criticized for a steep learning curve and for architectural churn, with recipes migrated to a separate submodule and evolving backend APIs causing version mismatches in older examples.[5](https://starlog.is/articles/llm-engineering/verl-project-verl/) Claims that DeepSeek's R1 reproduction projects (Open-R1, R1-V, TinyZero) converged on verl are not supported by any source in the retrieved record and are not asserted here.
## Recent developments
The framework's own arc is visible in the record: it began (late 2024) as an RLHF framework organized around the HybridFlow design, and by v0.7 it had moved toward RLVR-style reasoning training and agentic RL, with a rollout server, an AgentLoop abstraction for tool-using agents, and fully asynchronous trainer/rollout decoupling aimed at long chain-of-thought generation.[1](https://github.com/verl-project/verl?tab=readme-ov-file) • [4](https://github.com/verl-project/verl/blob/main/docs/blog/v0.7.md)
## References
1. [verl-project/verl README](https://github.com/verl-project/verl?tab=readme-ov-file)
2. [verl official documentation](https://verl.readthedocs.io/en/latest/)
3. [verl v0.7.1 on PyPI](https://pypi.org/project/verl/0.7.1/)
4. [verl v0.7 release blog](https://github.com/verl-project/verl/blob/main/docs/blog/v0.7.md)
5. [VERL: The Hybrid-Controller Framework Reshaping How We Train LLMs with Reinforcement Learning (Starlog)](https://starlog.is/articles/llm-engineering/verl-project-verl/)
---
*Topic: Encyclopedia › Technology and the built world › Computing and digital systems › Modern AI: foundation models, generative AI, and the AI industry › Foundation-model methods and training › Post-training and alignment methods*
*Initially written Sep 17, 2026 · Edited: Oct 11, 2026; Oct 15, 2026*
*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*
License: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license