AgileRL

The complete post-training stack for LLMs, plus RL for any problem. SFT, DPO and multi-turn agentic RL, with evolutionary hyperparameter optimization built in.

What is AgileRL?

The complete post-training stack for LLMs, plus RL for any problem. SFT, DPO and multi-turn agentic RL, with evolutionary hyperparameter optimization built in.

Overview of AgileRL

What it is

AgileRL is an open-source library that provides a complete post-training stack for large language models (LLMs) and classic reinforcement learning (RL) algorithms. It includes evolutionary hyperparameter optimization (HPO) built in, and supports training from supervised fine-tuning through preference tuning to multi-turn agentic RL. The library also covers classic deep RL for single-agent, multi-agent, offline, and contextual bandit problems. It is licensed under Apache-2.0 and is available on GitHub.

Key features

  • Full LLM post-training pipeline: SFT, DPO, and RL with GRPO, CISPO, GSPO, REINFORCE, or PPO.
  • Multi-turn agentic RL: train on any OpenEnv environment, a dataset plus reward function, or a remote env server.
  • LoRA and QLoRA support for memory-efficient fine-tuning.
  • Torch-native parallelism with data parallel or FSDP2 sharding, launched via torchrun.
  • Memory optimizations including chunked fused log-probs, activation checkpointing and offload, CPU offload for optimizer state or parameters, and trainer offload during rollout.
  • Model-specific optimizations for dense, MoE, and hybrid Mamba models (e.g., MoE LoRA per expert, fused kernels for Nemotron-H).
  • Fast rollouts using vLLM generation, with optional colocation of trainer and vLLM on a single GPU.
  • Evolutionary HPO for LLMs: hyperparameters like learning rate and KL penalty tune themselves during the run.
  • Agent-friendly CLI: training runs defined by a single YAML manifest and one command.
  • Async training at scale on Arena (managed GPU clusters) with models up to 120B parameters.
  • Deploy and chat: deploy the best checkpoint on Arena and interact via CLI.
  • Classic RL algorithms (on-policy, off-policy, offline, multi-agent, contextual bandits) all with evolutionary HPO.

Use cases

  • Post-training of open-weights LLMs: from SFT and DPO to multi-turn agentic RL.
  • Training and deploying RL agents for single-agent tasks (e.g., Gymnasium environments), multi-agent tasks (e.g., PettingZoo), offline RL, and contextual bandits.
  • Scaling LLM post-training to large models (up to 120B) on managed GPU clusters via Arena.
  • Rapid hyperparameter tuning without separate trial runs, using population-based evolutionary HPO.

Getting started

bash pip install agilerl # classic RL + Arena CLI pip install agilerl[llm] # LLM post-training For development tip of main branch: pip install git+https://github.com/AgileRL/AgileRL.git@main. For development mode: clone the repo and run pip install -e ..

License and pricing

License: Apache-2.0. Pricing: open source (no stated cost).

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/agilerlagilerl.svg)](https://yunodb.com/tools/agilerlagilerl?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/agilerlagilerl?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/agilerlagilerl.svg" alt="Featured on YunoDB" height="20"></a>