reasoning-from-scratch
Implement a reasoning LLM in PyTorch from scratch, step by step
What is reasoning-from-scratch?
Implement a reasoning LLM in PyTorch from scratch, step by step
Overview of reasoning-from-scratch
What it is
This repository contains the official code for the book Build a Reasoning Model (From Scratch) by Sebastian Raschka. It provides a hands-on, step-by-step implementation of a reasoning large language model (LLM) in PyTorch, starting from a pre-trained base LLM and adding reasoning capabilities in code. The methods mirror approaches used in large-scale reasoning models such as DeepSeek R1 and GPT-5 Thinking.
Key features
- Step-by-step code to add reasoning capabilities to a pre-trained base LLM
- Covers inference-time scaling, self-refinement, reinforcement learning (GRPO), and distillation
- Includes code for loading weights of existing, pretrained models
- Designed to run on consumer hardware for most chapters; GPU recommended for chapters 5 and 6
- Provides exercise solutions and bonus materials in each chapter folder
- Companion book to Build a Large Language Model (From Scratch)
Use cases
- Learning how reasoning LLMs work through practical implementation
- Educational development of small-but-functional reasoning models
- Understanding inference-time scaling, reinforcement learning, and distillation techniques
- Evaluating and improving LLM reasoning performance
Getting started
To download a copy of this repository, run the following command in your terminal:bashgit clone --depth 1 https://github.com/rasbt/reasoning-from-scratch.gitChapter 2 of the book provides additional tips on installing Python, managing Python packages, and setting up your coding environment.
License and pricing
This project is licensed under the Apache-2.0 license. The book is published by Manning and has ISBN 9781633434677.
Based on the project's public README and repository metadata.