any-llm

Communicate with an LLM provider using a single interface

What is any-llm?

Communicate with an LLM provider using a single interface

Overview of any-llm

What it is

any-llm is a Python SDK that provides a single, unified interface for communicating with multiple LLM providers. It supports OpenAI, Anthropic, Azure / Microsoft Foundry, Mistral, Ollama, and others. The project is maintained by Mozilla AI and is licensed under Apache-2.0. It is designed to let developers switch between providers without changing their code, and it leverages official provider SDKs for compatibility. The SDK is framework-agnostic and can be used across different projects and use cases. It also offers a migration path for users coming from LiteLLM, allowing them to keep their existing API keys and environment variables. The project includes a gateway component called Otari for budget management, API key management, usage analytics, and multi-tenant support. The SDK requires Python 3.11 or newer and API keys for the chosen providers. It supports both direct API functions for quick experimentation and a class-based approach for production applications. The SDK also supports streaming, tools, and the OpenAI-style Responses API. The project's documentation includes a full API reference, a list of supported providers, and cookbook examples. The project welcomes contributions from developers of all skill levels.

Key features

  • Single, unified interface for multiple LLM providers - Supports OpenAI, Anthropic, Azure / Microsoft Foundry, Mistral, Ollama, and more - Leverages official provider SDKs for maximum compatibility - Full type hints for better IDE support and clear error messages - Framework-agnostic design - Supports streaming, tools, and Responses API - Offers both direct API functions and a class-based approach - Migration path from LiteLLM with no proxy or extra config needed

Use cases

  • Switching between different LLM providers without code changes - Building applications that need to support multiple LLM backends - Experimenting with different models in scripts or notebooks - Production applications that require connection pooling and reuse of providers - Using OpenAI-compatible gateways or local servers that are not listed as dedicated providers

Getting started

python pip install 'any-llm-sdk[mistral,ollama]' export MISTRAL_API_KEY='YOUR_KEY_HERE' from any_llm import completion import os assert os.environ.get('MISTRAL_API_KEY') response = completion( model='mistral-small-latest', provider='mistral', messages=[{'role': 'user', 'content': 'Hello!'}] ) print(response.choices[0].message.content)

License and pricing

The project is licensed under the Apache License 2.0. The pricing type is open source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/mozilla-aiany-llm.svg)](https://yunodb.com/tools/mozilla-aiany-llm?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/mozilla-aiany-llm?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/mozilla-aiany-llm.svg" alt="Featured on YunoDB" height="20"></a>