mlx-vlm

MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.

What is mlx-vlm?

MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.

Overview of mlx-vlm

What it is

MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on Mac computers with Apple Silicon, using the MLX framework. It supports text, images, video, and audio inputs. The project is built by Neywa Labs.

Key features

  • Supports inference and fine-tuning of multimodal models (text, image, video, audio).
  • Provides a command-line interface (CLI) for generation tasks.
  • Includes a Gradio-based chat UI (requires extra dependency).
  • Offers a FastAPI server with continuous batching, automatic prefix caching (APC), and KV cache quantization.
  • Supports speculative decoding with DFlash, DFlash2, DSpark, Gemma 4 MTP, Gemma 4 EAGLE-3, and MiniMax M3 EAGLE-3 drafter families.
  • Includes agent skills bundles for common workflows (CLI inference, server inference, conversion/quantization, adding new models, benchmarking, contributing, and reproducible GitHub issues).
  • Provides model-specific documentation for many supported models (e.g., LFM3, DeepSeek-OCR, Phi-4, Gemma 4, MiniCPM, etc.).

Use cases

  • Running vision-language model inference on Apple Silicon Macs.
  • Fine-tuning VLMs for custom tasks.
  • Building local multimodal AI applications (chat, OCR, audio understanding, image generation, 3D Gaussian splats).
  • Deploying a local server for multimodal inference with continuous batching and caching.
  • Experimenting with speculative decoding to speed up generation.

Getting started

Install the package using pip:shpip install -U mlx-vlmFor the Gradio chat UI, install with the ui extra:shpip install -U 'mlx-vlm[ui]'

License and pricing

The project is licensed under the MIT License. It is open source and free to use.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/neywa-labsmlx-vlm.svg)](https://yunodb.com/tools/neywa-labsmlx-vlm?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/neywa-labsmlx-vlm?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/neywa-labsmlx-vlm.svg" alt="Featured on YunoDB" height="20"></a>