runanywhere-sdks
Production ready toolkit to run AI locally
What is runanywhere-sdks?
Production ready toolkit to run AI locally
Overview of runanywhere-sdks
What it is
RunAnywhere is a production-ready toolkit for running AI locally on phones, browsers, desktops, and servers. It provides eight SDKs built on a single C++ core, with a capability registry that routes each call to the best engine available on the device. The toolkit supports LLMs, vision, speech, voice agents, RAG, embeddings, and image generation, all running on-device with privacy by default and offline by design.
Key features
- LLM chat with token streaming, multi-turn history, and LoRA adapters (models include Llama, Qwen, Gemma, Phi, LFM, SmolLM, DeepSeek, and more)
- Structured output with schema-validated JSON and constrained decoding
- Tool calling with local function tools and an agent loop (parallel calls when supported)
- Vision (VLM) for image understanding, live camera description, and photo Q&A
- Speech-to-Text (Whisper, Moonshine) and Text-to-Speech (Piper, Kokoro, Kitten, MeloTTS, Magpie)
- Voice agents combining VAD, STT, LLM, and TTS in one pipeline
- Embeddings (L2-normalized vectors), RAG (local document ingestion and retrieval-augmented answers), and image generation (Stable Diffusion on Core ML, inpainting on Hexagon NPU)
Use cases
- Running LLMs, vision, and speech entirely on-device for privacy-sensitive applications
- Building voice agents that work offline on mobile or desktop
- Local document search and retrieval-augmented generation (RAG)
- Generating images on-device using diffusion models
- Deploying AI across multiple platforms (iOS, Android, Web, Desktop) with a single semantic API
Getting started
Install the Python package:
pip install runanywhere
Then use in Python:
import runanywhere as ra
from runanywhere import LlmOptions
ra.initialize()
print(ra.llm.generate("Explain on-device AI in one sentence.",
LlmOptions(model="qwen2.5-0.5b")).text)
For a CLI alternative, install Wally from RunanywhereAI/wally. For platform-specific SDKs (Swift, Kotlin, Flutter, React Native, Web, Electron), see the repository README.
License and pricing
The license is not stated in the repository metadata (NOASSERTION). The project is listed as open source.
Based on the project's public README and repository metadata.