mlx-serve

Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, vide

What is mlx-serve?

Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation.

Overview of mlx-serve

What it is

mlx-serve is a native Zig server that runs any LLM on Apple Silicon, supporting both MLX-format models and GGUF models from HuggingFace. It exposes OpenAI-compatible and Anthropic-compatible HTTP APIs, as well as the Ollama API, on a single port. Beyond text, the same server can generate images, video, music, speech with voice cloning, and 3D models natively on MLX. It ships with MLX Core, a macOS menu-bar app that provides chat, agent mode, MCP tool calling, and model management. No Python is required at runtime.

Key features

  • Supports any LLM: all supported MLX architectures plus the entire GGUF universe via embedded llama.cpp, including DeepSeek V4 Flash through a dedicated engine.
  • Four API surfaces on one port: OpenAI chat/completions and Responses (with WebSocket transport), Anthropic Messages, and the Ollama API.
  • Media generation: image, video, music, speech with voice cloning, and 3D model generation, all natively on MLX.
  • Built-in macOS menu-bar app with chat, agent mode, MCP tool calling, and model management.
  • Performance optimizations: speculative decoding (PLD, drafter, native MTP), custom Metal kernels, continuous batching, KV-cache quantization, prefix and tokenize caches.
  • Works with coding agents: Claude Code, pi, OpenCode, and others, with one-click launchers from the app or terminal.
  • LAN model sharing: use another Mac's models over Bonjour with zero configuration.

Use cases

  • Running any LLM locally on Apple Silicon for inference, chat, or API access.
  • Using local models with coding agents and editors that support OpenAI, Anthropic, or Ollama APIs.
  • Generating images, video, music, speech, and 3D models from a single server.
  • Sharing models across multiple Macs on a local network without manual setup.

Getting started

Needs macOS 26.2+ on Apple Silicon. Install via Homebrew:bashbrew tap ddalcu/mlx-serve https://github.com/ddalcu/mlx-servebrew trust ddalcu/mlx-servebrew install --cask mlx-serve # the app (recommended)brew install mlx-serve # CLI + server only, no GUIOr use the terminal directly:bashmlx-serve run gemma4 # downloads and serves Gemma 4, chats in terminalmlx-serve pull qwen3.6:27b # download a modelmlx-serve serve # serve all pulled modelsFor Linux (NVIDIA), download the appropriate tarball from releases and run zig-out/bin/mlx-serve. See the repository README for build instructions.

License and pricing

License: MIT. Pricing: open source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/ddalcumlx-serve.svg)](https://yunodb.com/tools/ddalcumlx-serve?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/ddalcumlx-serve?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/ddalcumlx-serve.svg" alt="Featured on YunoDB" height="20"></a>