LocalAI

LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

What is LocalAI?

LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

Overview of LocalAI

What it is

LocalAI is an open-source AI engine that allows running any model—LLMs, vision, voice, image, video—on any hardware without requiring a GPU. It is designed with a small core where each backend wraps a best-in-class engine (e.g., llama.cpp, vLLM, whisper.cpp, stable-diffusion, MLX) in its own image, pulled only when a model needs it.

Key features

  • Composable by design: backends are separate and pulled on demand, so you install only what your model needs - Open and extensible: load any model, or build your own backend in any language against an open interface - Drop-in API compatibility: OpenAI, Anthropic, and ElevenLabs APIs across every backend - Any model, any modality: LLMs, vision, voice, image, and video behind one API - Any hardware: NVIDIA, AMD, Intel, Apple Silicon, Vulkan, or CPU-only - Multi-user ready: API key auth, user quotas, role-based access - Privacy-first: your data never leaves your infrastructure

Use cases

Running large language models, vision models, speech recognition, image generation, and video processing locally on any hardware without a GPU. Suitable for privacy-sensitive applications where data must remain on-premises. Can be used for multi-user environments with authentication and quotas.

Getting started

bash # CPU only: docker run -ti --name local-ai -p 8080:8080 localai/localai:latest # NVIDIA GPU: docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-13 # Loading a model from the gallery: local-ai run llama-3.2-1b-instruct:q4_k_m # Loading a model from Huggingface: local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf

License and pricing

Licensed under MIT. Open-source and free to use.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/mudlerlocalai.svg)](https://yunodb.com/tools/mudlerlocalai?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/mudlerlocalai?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/mudlerlocalai.svg" alt="Featured on YunoDB" height="20"></a>