Study-Reinforcement-Learning
RL study guide — foundations through RLHF, DPO, GRPO, RLVR, agentic RL, and offline RL. Hand-written CS294 notes, 34 lecture drafts, 9 tested exercises, citatio
What is Study-Reinforcement-Learning?
RL study guide — foundations through RLHF, DPO, GRPO, RLVR, agentic RL, and offline RL. Hand-written CS294 notes, 34 lecture drafts, 9 tested exercises, citations that resolve.
Overview of Study-Reinforcement-Learning
What it is
Study-Reinforcement-Learning is a personal study repository that serves as a guide for learning reinforcement learning (RL) and its application to training language models. It covers topics from Markov decision processes and policy gradients through RLHF, DPO, GRPO, RLVR, agentic RL, and offline RL. The repository includes hand-written notes from a 2017 Berkeley CS 294 course, 34 lecture drafts, 9 tested coding exercises, and curated external references. It is not a library but a collection of study materials.
Key features
- Hand-written student notes from CS 294 Deep RL (Berkeley, Spring 2017) by Levine, Schulman, and Finn, kept as originally written.
- A 34-lecture series (unreviewed, treat with skepticism) covering MDPs through RLHF, DPO, GRPO, RLVR, agentic RL, and offline RL.
- Nine small coding exercises with pytest tests and reference solutions, verified to pass. Topics include value iteration, REINFORCE, Q-learning, actor-critic, PPO, Bradley-Terry reward model, DPO, GRPO, and RND exploration.
- Curated external links: talks (Pineau, Abbeel), books (Sutton & Barto, Szepesvári), courses (David Silver, CS285, Hugging Face Deep RL Course, Spinning Up).
- Tools: arxiv-collector (fetches arXiv papers), lit-builder (ICLR/NeurIPS/ICML triage with keyword filter and LLM scoring), content-pipeline (drafts blog posts from papers).
- A website at rl.yad.codes that hosts the lectures, exercises, notes, and a Q-learning agent that can be watched learn in a browser.
Use cases
- New to RL: Start with the talks, books, and courses listed in the README (Pineau's intro, Sutton & Barto, David Silver's UCL course, or CS285).
- Want hands-on practice: Complete the nine coding exercises in the
exercises/directory. They are tested and run. - Curious about modern LLM RL: Read the 34-lecture series in
notes/lectures/covering RLHF, DPO, GRPO, RLVR, agentic, and offline RL. Cross-check claims against cited papers. - Working with Claude Code or Codex: Read
AGENTS.mdfirst for repository organization and workflow.
Getting started
See the repository README for guidance on getting started. It recommends starting with the talks/books/courses if new to RL, doing the exercises for hands-on practice, and reading the lecture series for modern LLM RL topics.
License and pricing
The license is not stated (NOASSERTION). The repository is open source.
Based on the project's public README and repository metadata.