mlx-serve
Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, vide
What is mlx-serve?
Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation.
Overview of mlx-serve
What it is
mlx-serve is a native Zig server that runs any LLM on Apple Silicon, supporting both MLX-format models and GGUF models from HuggingFace. It exposes OpenAI-compatible and Anthropic-compatible HTTP APIs, as well as the Ollama API, on a single port. Beyond text, the same server can generate images, video, music, speech with voice cloning, and 3D models natively on MLX. It ships with MLX Core, a macOS menu-bar app that provides chat, agent mode, MCP tool calling, and model management. No Python is required at runtime.
Key features
- Supports any LLM: all supported MLX architectures plus the entire GGUF universe via embedded llama.cpp, including DeepSeek V4 Flash through a dedicated engine.
- Four API surfaces on one port: OpenAI chat/completions and Responses (with WebSocket transport), Anthropic Messages, and the Ollama API.
- Media generation: image, video, music, speech with voice cloning, and 3D model generation, all natively on MLX.
- Built-in macOS menu-bar app with chat, agent mode, MCP tool calling, and model management.
- Performance optimizations: speculative decoding (PLD, drafter, native MTP), custom Metal kernels, continuous batching, KV-cache quantization, prefix and tokenize caches.
- Works with coding agents: Claude Code, pi, OpenCode, and others, with one-click launchers from the app or terminal.
- LAN model sharing: use another Mac's models over Bonjour with zero configuration.
Use cases
- Running any LLM locally on Apple Silicon for inference, chat, or API access.
- Using local models with coding agents and editors that support OpenAI, Anthropic, or Ollama APIs.
- Generating images, video, music, speech, and 3D models from a single server.
- Sharing models across multiple Macs on a local network without manual setup.
Getting started
Needs macOS 26.2+ on Apple Silicon. Install via Homebrew:bashbrew tap ddalcu/mlx-serve https://github.com/ddalcu/mlx-servebrew trust ddalcu/mlx-servebrew install --cask mlx-serve # the app (recommended)brew install mlx-serve # CLI + server only, no GUIOr use the terminal directly:bashmlx-serve run gemma4 # downloads and serves Gemma 4, chats in terminalmlx-serve pull qwen3.6:27b # download a modelmlx-serve serve # serve all pulled modelsFor Linux (NVIDIA), download the appropriate tarball from releases and run zig-out/bin/mlx-serve. See the repository README for build instructions.
License and pricing
License: MIT. Pricing: open source.
Based on the project's public README and repository metadata.