mesh-llm

Distributed AI/LLM for the people. Share compute privately or publicly to power your agents and chat.

What is mesh-llm?

Distributed AI/LLM for the people. Share compute privately or publicly to power your agents and chat.

Overview of mesh-llm

What it is

Mesh LLM pools GPUs and memory across machines and exposes the result as one OpenAI-compatible API at http://localhost:9337/v1. It allows starting one node and adding more later, with the mesh deciding whether a model runs locally, routes to a peer, or uses Skippy stage splits for models too large for one box.

Key features

  • Single-machine fit first: if one node can host the full model, it serves locally without stage traffic.
  • Mesh routing: every node exposes the same /v1 API; requests are routed by the model field to the peer that can serve that model.
  • Encrypted peer transport: QUIC end-to-end encrypts traffic between Mesh nodes; Iroh relays forward encrypted packets without reading payloads.
  • Owner-control plane: operator config and inventory actions use an additive mesh-llm-control/2 lane with explicit endpoint bootstrap.
  • Skippy stage splits: large dense models can load as package-backed layer stages; the coordinator plans contiguous layer ranges.
  • Layer packages: package repositories contain model-package.json plus GGUF fragments so peers fetch only needed pieces.
  • Public discovery: published meshes advertise through Nostr discovery; private meshes stay invite-token based.

Use cases

  • Running large language models that do not fit on a single machine by splitting across multiple nodes.
  • Creating private or public meshes for collaborative AI inference.
  • Integrating with agent tools like Goose, OpenCode, Claude Code, or Pi via mesh-llm subcommands.
  • Debugging and development with local model-only serving (bypassing mesh networking).

Getting started

Install the latest release executable:

curl -fsSL https://meshllm.cloud/install.sh | bash

On Windows, use PowerShell:

irm https://meshllm.cloud/install.ps1 | iex

Install the Apple Silicon Homebrew formula with brew install Mesh-LLM/tap/mesh-llm. Finish setup:

mesh-llm setup

Join the public mesh and start serving:

mesh-llm serve --auto

License and pricing

Licensed under Apache-2.0. The project is open source (pricing_type: open_source). No pricing information is stated beyond the open source license.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/mesh-llmmesh-llm.svg)](https://yunodb.com/tools/mesh-llmmesh-llm?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/mesh-llmmesh-llm?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/mesh-llmmesh-llm.svg" alt="Featured on YunoDB" height="20"></a>