llama-swap

Reliable model swapping for any local AI server. Chosen by thousands of users. Batteries included. If we don't have it, you probably don't need it.

What is llama-swap?

Reliable model swapping for any local AI server. Chosen by thousands of users. Batteries included. If we don't have it, you probably don't need it.

Overview of llama-swap

What it is

llama-swap is a tool that allows running multiple generative AI models on a local machine and hot-swapping between them on demand. It works with any OpenAI and Anthropic API compatible server. It is built in Go, has zero dependencies, and consists of one binary and one configuration file.

Key features

  • On-demand model switching for many local AI servers (llama.cpp + forks, vllm, stable-diffusion.cpp, audio.cpp, ComfyUI, etc.)
  • Supports OpenAI API endpoints: v1/completions, v1/chat/completions, v1/responses, v1/embeddings, v1/models, v1/audio/speech, v1/audio/transcriptions, v1/audio/voices, v1/realtime, v1/images/generations, v1/images/edits - Supports Anthropic API endpoints: v1/messages, v1/messages/count_tokens - Supports llama-server (llama.cpp) endpoints: v1/rerank, v1/reranking, /rerank, /infill, /completion, /models, /props - Supports SDAPI via stable-diffusion.cpp's server: /sdapi/v1/txt2img, /sdapi/v1/img2img, /sdapi/v1/loras - Includes a web UI with a playground, token metrics, request/response inspection, manual model load/unload, and real-time log streaming - Provides API key support, runtime profile switching, concurrent model support with a DSL swap matrix, automatic model unloading after timeout (ttl), preloading models on startup (hooks), and request filters (stripParams, setParams, setParamsByID)

Use cases

  • Running multiple generative AI models on a single machine and switching between them without restarting the server - Powering local AI workflows that require different models for different tasks - Testing and comparing different models in a local environment - Integrating with tools that expect OpenAI or Anthropic compatible APIs

Getting started

shell # Docker install (unified container with CUDA 13) $ docker pull ghcr.io/mostlygeek/llama-swap:unified-cuda13 # run with a custom configuration and models directory $ docker run -it --rm --runtime nvidia -p 9292:8080 -v /path/to/models:/models -v /path/to/custom/config.yaml:/etc/llama-swap/config/config.yaml ghcr.io/mostlygeek/llama-swap:unified-cuda13 Other installation methods include Homebrew (macOS and Linux), MacPorts (macOS), WinGet, release binaries, and building from source. See the repository README for details.

License and pricing

The license is MIT. The project is open source and free to use.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/mostlygeekllama-swap.svg)](https://yunodb.com/tools/mostlygeekllama-swap?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/mostlygeekllama-swap?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/mostlygeekllama-swap.svg" alt="Featured on YunoDB" height="20"></a>