awesome-local-llm
A curated list of awesome platforms, tools, practices and resources that helps run LLMs locally
What is awesome-local-llm?
A curated list of awesome platforms, tools, practices and resources that helps run LLMs locally
Overview of awesome-local-llm
What it is
This is a curated list of platforms, tools, practices and resources that help run large language models (LLMs) locally. The repository is maintained on GitHub and serves as a directory of software and hardware related to local LLM inference, training, and deployment.
Key features
- Covers inference platforms, inference engines, user interfaces, large language models, tools, hardware, tutorials, and communities.
- Lists specific model providers such as Qwen, Mistral AI, Tencent, Unsloth AI, and bartowski.
- Includes sections for model explorers, benchmarks, and leaderboards.
- Provides references to specific models categorized by purpose: general purpose, coding, decision, multimodal, image, audio, retrieval-augmented generation, safeguards, and miscellaneous.
- Contains tool categories for models, agent frameworks, model context protocol, retrieval-augmented generation, coding agents, computer use, browser automation, memory management, testing, evaluation and observability, research, training and fine-tuning, security and sandboxing, and miscellaneous.
- Lists hardware resources and tutorials on models, prompt engineering, context engineering, inference, agents, retrieval-augmented generation, and miscellaneous topics.
Use cases
This list is intended for developers, researchers, and enthusiasts who want to discover and compare software and hardware options for running LLMs locally. It can be used to find inference engines like Ollama or llama.cpp, user interfaces like Open WebUI, or specific models for tasks such as coding, multimodal processing, or retrieval-augmented generation.
Getting started
See the repository README for the full table of contents and links to each listed resource.
License and pricing
The repository is licensed under MIT. The pricing type is listed as open source.
Based on the project's public README and repository metadata.