EasyR1
EasyR1: An Efficient, Scalable, Multi-Modality RL Training Framework based on veRL
What is EasyR1?
EasyR1: An Efficient, Scalable, Multi-Modality RL Training Framework based on veRL
Overview of EasyR1
What it is
EasyR1 is an efficient, scalable, multi-modality reinforcement learning training framework based on veRL. It is a clean fork of the original veRL project to support vision language models. The framework is designed to be efficient and scalable due to the design of HybirdEngine and the latest release of vLLM's SPMD mode.
Key features
- Supported models include Llama3, Qwen2, Qwen2.5, Qwen3 language models, Qwen2-VL, Qwen2.5-VL, Qwen3-VL vision language models, and DeepSeek-R1 distill models.
- Supported algorithms include GRPO, DAPO, Reinforce++, ReMax, RLOO, GSPO, and CISPO.
- Supports any text or vision-text dataset in a specific format.
- Supported tricks include padding-free training, LoRA training, resuming from the latest or best checkpoint, and tracking with Wandb, SwanLab, Mlflow, and Tensorboard.
- Provides a Dockerfile and pre-built Docker image for easy environment setup.
Use cases
EasyR1 can be used for reinforcement learning training of language models and vision language models on text and image-text datasets. It is suitable for tasks such as mathematical reasoning (e.g., Geometry3K dataset), counting problems (e.g., CLEVR-70k-Counting), and GeoQA problems. The framework also supports multi-node training for models larger than 70B parameters.
Getting started
bashgit clone https://github.com/hiyouga/EasyR1.gitcd EasyR1pip install -e .For Docker environment:bashdocker pull hiyouga/verl:ngc-th2.8.0-cu12.9-vllm0.11.0docker run -it --ipc=host --gpus=all hiyouga/verl:ngc-th2.8.0-cu12.9-vllm0.11.0
License and pricing
The project is licensed under Apache-2.0. It is open source with no stated pricing.
Based on the project's public README and repository metadata.