LocalAI
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.
What is LocalAI?
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.
Overview of LocalAI
What it is
LocalAI is an open-source AI engine that allows running any model—LLMs, vision, voice, image, video—on any hardware without requiring a GPU. It is designed with a small core where each backend wraps a best-in-class engine (e.g., llama.cpp, vLLM, whisper.cpp, stable-diffusion, MLX) in its own image, pulled only when a model needs it.
Key features
- Composable by design: backends are separate and pulled on demand, so you install only what your model needs - Open and extensible: load any model, or build your own backend in any language against an open interface - Drop-in API compatibility: OpenAI, Anthropic, and ElevenLabs APIs across every backend - Any model, any modality: LLMs, vision, voice, image, and video behind one API - Any hardware: NVIDIA, AMD, Intel, Apple Silicon, Vulkan, or CPU-only - Multi-user ready: API key auth, user quotas, role-based access - Privacy-first: your data never leaves your infrastructure
Use cases
Running large language models, vision models, speech recognition, image generation, and video processing locally on any hardware without a GPU. Suitable for privacy-sensitive applications where data must remain on-premises. Can be used for multi-user environments with authentication and quotas.
Getting started
bash # CPU only: docker run -ti --name local-ai -p 8080:8080 localai/localai:latest # NVIDIA GPU: docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-13 # Loading a model from the gallery: local-ai run llama-3.2-1b-instruct:q4_k_m # Loading a model from Huggingface: local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
License and pricing
Licensed under MIT. Open-source and free to use.
Based on the project's public README and repository metadata.