Soup

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

What is Soup?

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

Overview of Soup

What it is

Soup is a command-line application for fine-tuning and post-training large language models. It uses a single YAML configuration file. Layer streaming allows training an 8B model on a 4 GB laptop GPU. Measured on an RTX 3050 Laptop 4 GB: Llama-3.1-8B-Instruct + NF4 at 119.6 tok/s at 952 MHz, 3.32 GB peak (pre-repair, v0.72.2). Post-repair (v0.73.0) on another RTX 3050 Laptop 4 GB: 208.6 tok/s at 1935–1957 MHz in 2.40 GB peak. These figures are not directly comparable due to clock and memory pattern changes. Layer streaming is opt-in (stream_layers: true) and still BETA.

Key features

  • One YAML configuration file for the entire training setup.
  • No SSH required; runs locally on the user's GPU.
  • Automatic batch size, GPU detection, and quantization.
  • Layer streaming (BETA) to reduce VRAM usage.
  • Web UI for a local dashboard (soup ui).
  • Supports various training methods: LoRA, QLoRA, and others.
  • Templates for chat, code, tool-calling, medical, reasoning, vision, and more.

Use cases

Fine-tuning LLMs for specific tasks such as chat, code generation, tool calling, medical applications, reasoning, vision, and audio. Post-training with preference optimization (DPO, ORPO, SimPO, etc.). Also suitable for pretraining, long context, embedding, and MoE models.

Getting started

Install Soup with pipx or pip. The light core includes CLI and data tools; add [train] for the training stack. bash pipx install soup-cli pipx install 'soup-cli[train]' pip install soup-cli pip install 'soup-cli[train]' Create a config and train: bash soup init --template chat soup train For the Web UI: bash pip install 'soup-cli[ui]' soup ui

License and pricing

License: Apache-2.0. Pricing: open source (free).

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/makazhanalpamyssoup.svg)](https://yunodb.com/tools/makazhanalpamyssoup?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/makazhanalpamyssoup?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/makazhanalpamyssoup.svg" alt="Featured on YunoDB" height="20"></a>