turbo-fieldfare
Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook
What is turbo-fieldfare?
Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook
Overview of turbo-fieldfare
What it is
TurboFieldfare is a custom Swift and Metal runtime for running the Gemma 4 26B-A4B instruction-tuned model on Apple Silicon Macs. It is designed to keep memory usage around 2 GB by streaming only the required experts from SSD, rather than loading the full 14.3 GB model into RAM. The project includes a native Mac app, a command-line interface, and an experimental OpenAI-compatible server. It is model-specific and not a wrapper around MLX or llama.cpp.
Key features
- Runs Gemma 4 26B-A4B (26B total parameters, ~3.88B active per token) on any M-series Mac, including 8 GB models.
- Keeps ~2 GB of weights and 4K KV cache in memory; streams experts from SSD as needed.
- Uses MLX affine 4-bit quantization (group 64), 8-bit router, and 4-bit shared and routed experts.
- Provides a native Mac app, CLI, and loopback OpenAI-compatible Chat Completions server.
- Supports optional image vision tower for M2 or newer Macs (text-only on M1).
- Includes a streaming installer that repacks the model directly from Hugging Face without materializing the full checkpoint.
Use cases
- Running a large language model locally on a Mac with limited RAM (e.g., 8 GB).
- Instruction following and chat conversations with a 26B-parameter model.
- Experimenting with MoE (Mixture of Experts) inference on Apple Silicon hardware.
- Local development and testing of LLM applications without cloud dependencies.
Getting started
bash git clone https://github.com/drumih/turbo-fieldfare.git cd turbo-fieldfare swift build -c release .build/release/TurboFieldfareMac On first run, the app will prompt to download the model (about 15 GB). After installation, choose Load Model, type a prompt, and press Generate.
License and pricing
The project is licensed under Apache-2.0 and is open source.
Based on the project's public README and repository metadata.