any-llm
Communicate with an LLM provider using a single interface
What is any-llm?
Communicate with an LLM provider using a single interface
Overview of any-llm
What it is
any-llm is a Python SDK that provides a single, unified interface for communicating with multiple LLM providers. It supports OpenAI, Anthropic, Azure / Microsoft Foundry, Mistral, Ollama, and others. The project is maintained by Mozilla AI and is licensed under Apache-2.0. It is designed to let developers switch between providers without changing their code, and it leverages official provider SDKs for compatibility. The SDK is framework-agnostic and can be used across different projects and use cases. It also offers a migration path for users coming from LiteLLM, allowing them to keep their existing API keys and environment variables. The project includes a gateway component called Otari for budget management, API key management, usage analytics, and multi-tenant support. The SDK requires Python 3.11 or newer and API keys for the chosen providers. It supports both direct API functions for quick experimentation and a class-based approach for production applications. The SDK also supports streaming, tools, and the OpenAI-style Responses API. The project's documentation includes a full API reference, a list of supported providers, and cookbook examples. The project welcomes contributions from developers of all skill levels.
Key features
- Single, unified interface for multiple LLM providers - Supports OpenAI, Anthropic, Azure / Microsoft Foundry, Mistral, Ollama, and more - Leverages official provider SDKs for maximum compatibility - Full type hints for better IDE support and clear error messages - Framework-agnostic design - Supports streaming, tools, and Responses API - Offers both direct API functions and a class-based approach - Migration path from LiteLLM with no proxy or extra config needed
Use cases
- Switching between different LLM providers without code changes - Building applications that need to support multiple LLM backends - Experimenting with different models in scripts or notebooks - Production applications that require connection pooling and reuse of providers - Using OpenAI-compatible gateways or local servers that are not listed as dedicated providers
Getting started
python pip install 'any-llm-sdk[mistral,ollama]' export MISTRAL_API_KEY='YOUR_KEY_HERE' from any_llm import completion import os assert os.environ.get('MISTRAL_API_KEY') response = completion( model='mistral-small-latest', provider='mistral', messages=[{'role': 'user', 'content': 'Hello!'}] ) print(response.choices[0].message.content)
License and pricing
The project is licensed under the Apache License 2.0. The pricing type is open source.
Based on the project's public README and repository metadata.