rllm
Democratizing Reinforcement Learning for LLMs
What is rllm?
Democratizing Reinforcement Learning for LLMs
Overview of rllm
What it is
rLLM is an open-source framework for training language agents with reinforcement learning. It allows users to bring any harness, run it in any sandbox, and switch training backends with one flag, so the same agent code drives both evaluation and training.
Key features
- Supports 10+ CLI harnesses (Claude Code, Codex, Terminus-2, mini-swe-agent, opencode, etc.) and Harbor-compatible task directories. Users can wrap custom agents with
@rllm.rollout. - Sandbox options include Docker, Daytona, Modal, or local execution, with snapshot and warm-pool acceleration to reduce rollout costs at training scale.
- Multiple training backends:
verl(distributed multi-GPU),tinker(single-machine), andfireworks(Fireworks platform), switchable with one flag. - 60+ integrated benchmarks covering math, code, MCQ, QA, search, VLM, translation, and agentic tasks, including Terminal-Bench 2.0, SWE-bench, SkillsBench, AIME, MATH-500, and GPQA.
- Multiple training methods: GRPO, REINFORCE, RLOO, SFT, and on-policy distillation.
- Battle-tested with state-of-the-art open-source results (DeepScaleR-1.5B, DeepCoder-14B, DeepSWE-32B, FinQA-4B) and adopted by academic labs and industry research teams.
Use cases
rLLM is designed for researchers and developers who want to train language agents with reinforcement learning on any benchmark, using any agent harness and any sandbox environment. It is suitable for single-turn and multi-agent tasks, long-horizon terminal agents, multimodal deep research, and financial analysis, among other applications.
Getting started
rLLM requires Python >= 3.11. Install via pip: bash uv pip install 'rllm @ git+https://github.com/rllm-org/rllm.git' For distributed multi-GPU training: bash uv pip install 'rllm[verl] @ git+https://github.com/rllm-org/rllm.git' For Fireworks training platform: bash uv pip install 'rllm[fireworks] @ git+https://github.com/rllm-org/rllm.git' Quickstart CLI: bash rllm model setup rllm eval gsm8k rllm train gsm8k For Python API usage, see the repository README.
License and pricing
The project is licensed under Apache-2.0 and is open source. Pricing is not stated beyond being open source.
Based on the project's public README and repository metadata.