reasoning-from-scratch

Implement a reasoning LLM in PyTorch from scratch, step by step

What is reasoning-from-scratch?

Implement a reasoning LLM in PyTorch from scratch, step by step

Overview of reasoning-from-scratch

What it is

This repository contains the official code for the book Build a Reasoning Model (From Scratch) by Sebastian Raschka. It provides a hands-on, step-by-step implementation of a reasoning large language model (LLM) in PyTorch, starting from a pre-trained base LLM and adding reasoning capabilities in code. The methods mirror approaches used in large-scale reasoning models such as DeepSeek R1 and GPT-5 Thinking.

Key features

  • Step-by-step code to add reasoning capabilities to a pre-trained base LLM
  • Covers inference-time scaling, self-refinement, reinforcement learning (GRPO), and distillation
  • Includes code for loading weights of existing, pretrained models
  • Designed to run on consumer hardware for most chapters; GPU recommended for chapters 5 and 6
  • Provides exercise solutions and bonus materials in each chapter folder
  • Companion book to Build a Large Language Model (From Scratch)

Use cases

  • Learning how reasoning LLMs work through practical implementation
  • Educational development of small-but-functional reasoning models
  • Understanding inference-time scaling, reinforcement learning, and distillation techniques
  • Evaluating and improving LLM reasoning performance

Getting started

To download a copy of this repository, run the following command in your terminal:bashgit clone --depth 1 https://github.com/rasbt/reasoning-from-scratch.gitChapter 2 of the book provides additional tips on installing Python, managing Python packages, and setting up your coding environment.

License and pricing

This project is licensed under the Apache-2.0 license. The book is published by Manning and has ISBN 9781633434677.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/rasbtreasoning-from-scratch.svg)](https://yunodb.com/tools/rasbtreasoning-from-scratch?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/rasbtreasoning-from-scratch?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/rasbtreasoning-from-scratch.svg" alt="Featured on YunoDB" height="20"></a>