paddock

Native Rust inference server for open models on NVIDIA and Apple Silicon. OpenAI- and Anthropic-compatible APIs, GGUF + safetensors + MLX, FP8/NVFP4/MXFP4/Q8/Q4

What is paddock?

Native Rust inference server for open models on NVIDIA and Apple Silicon. OpenAI- and Anthropic-compatible APIs, GGUF + safetensors + MLX, FP8/NVFP4/MXFP4/Q8/Q4, built-in Studio

Overview of paddock

What it is

Paddock is an AI inference server written in Rust, designed for NVIDIA GPUs and Apple Silicon. It provides OpenAI- and Anthropic-compatible APIs and includes a built-in Studio for exploring and comparing models with artifact support. The engine is native, not a wrapper, and includes its own scheduler, paged KV cache, memory management, and CUDA and Metal kernels. Two binaries are provided: paddock-runner (the core model runner) and paddock (a manager that bundles the Studio for easier model downloading and running). On macOS, a native Swift app is also available.

Key features

  • Native Rust inference engine with custom CUDA and Metal kernels.
  • Supports NVIDIA GPUs (Blackwell, Ampere, Ada Lovelace) and Apple Silicon (macOS 26 or later).
  • Compatible with OpenAI and Anthropic API formats.
  • Supports multiple weight formats: GGUF, safetensors, and MLX (on Apple Silicon).
  • Supports quantizations: FP8, NVFP4, MXFP4, Q8_0, Q4_K_XL/Q4_K_L/Q4_K_M, UD-Q3_K_XL, UD-IQ3_XXS, and Bonsai's ternary PTQ1_0.
  • Built-in Studio for model exploration, comparison, and artifact support.
  • Paged KV cache and efficient memory management.

Use cases

  • Running open-source language models in production on single GPUs.
  • Exploring and comparing different models and quantizations via the built-in Studio.
  • Performing text generation, vision tasks, tool use, image generation, decision-making, tabular data analysis, object segmentation, document OCR, and speech processing using supported models.
  • Testing models against cloud endpoints via OpenRouter integration.

Getting started

See the repository README for installation instructions. On macOS, downloads include a native app (.dmg), command-line installer (.pkg), or manual archive (.tar.gz). On NVIDIA systems, only the NVIDIA driver is required.

License and pricing

The license is not stated in the repository metadata. The software is listed as open source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/truesparpaddock.svg)](https://yunodb.com/tools/truesparpaddock?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/truesparpaddock?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/truesparpaddock.svg" alt="Featured on YunoDB" height="20"></a>