uni-api
This is a project that unifies the management of LLM APIs. It can call multiple backend services through a unified API interface, convert them to the OpenAI for
What is uni-api?
This is a project that unifies the management of LLM APIs. It can call multiple backend services through a unified API interface, convert them to the OpenAI format uniformly, and support load balancing. Currently supported backend services include: OpenAI, Anthropic, DeepBricks, OpenRouter, Gemini, Vertex, etc.
Overview of uni-api
What it is
uni-api is an open-source project that provides a unified management layer for large language model (LLM) APIs. It allows users to call multiple backend services through a single API interface that conforms to the OpenAI format. The project is designed for personal use, offering a simpler alternative to more complex commercial solutions. It supports load balancing and automatic retries across backend providers.
Key features
- Supports multiple backend providers including OpenAI, Anthropic, Gemini, Vertex, Azure, AWS, xai, Cohere, Groq, Cloudflare, OpenRouter, and 0-0.pro.
- Provides a pure configuration file approach (no front-end) for setting up API channels.
- Offers three types of load balancing: channel-level weighted, channel-level sequential (round-robin), and automatic API key-level round-robin.
- Includes automatic retry and channel cooling: when a channel fails, it is temporarily excluded and cooled down.
- Supports fine-grained model timeout settings, permission control with wildcards, and rate limiting (e.g., 2/min, 5/hour).
- Exposes multiple standard OpenAI format endpoints: /v1/chat/completions, /v1/responses, /v1/images/generations, /v1/embeddings, /v1/audio/transcriptions, /v1/audio/speech, /v1/moderations, /v1/models.
- Supports native tool use function calls and image recognition for OpenAI, Anthropic, Gemini, Vertex, Azure, and AWS.
- Includes OpenAI moderation moral review to reduce the risk of backend API bans.
Use cases
- Personal use: running a private API station that aggregates multiple LLM providers behind a single OpenAI-compatible endpoint.
- Load balancing: distributing requests across multiple API keys or providers to improve reliability and throughput.
- Cost and permission management: controlling which models specific API keys can access and setting rate limits.
- Fallback and retry: automatically retrying failed requests on alternative channels to ensure service continuity.
Getting started
To start uni-api, you must provide a configuration file. There are two methods: 1. Use the CONFIG_URL environment variable to point to a configuration file URL, which will be downloaded at startup. 2. Mount a configuration file named api.yaml into the container. Example minimal api.yaml: ```yaml providers:
- provider: provider_name base_url: https://api.your.com/v1/chat/completions api: sk-YgS6GTi0b4bEabc4C api_keys:
- api: sk-Pkj60Yf8JFWxfgRmXQFWyGtWUddGZnmi3KlvowmRWpWpQxx ``` For detailed advanced configuration, see the repository README.
License and pricing
The project is licensed under Apache-2.0 and is open source. Pricing is not stated; the project is free to use under the license terms.
Based on the project's public README and repository metadata.