agent-router

Manages Unified Access to Generative AI Services built on Envoy Gateway

What is agent-router?

Manages Unified Access to Generative AI Services built on Envoy Gateway

Overview of agent-router

What it is

Agent Router is an open source control plane for AI and agent traffic, built on Envoy Gateway. It provides a unified, OpenAI-compatible API for accessing multiple generative AI models and tools, from hosted providers to self-hosted inference and MCP servers. The project was formerly known as Envoy AI Gateway and is now an Agentic AI Foundation project. It is licensed under Apache 2.0.

Key features

  • Provides one consistent, OpenAI-compatible API for every model and tool.
  • Centralizes credential management, routing, quotas, failover, and usage attribution for platform teams.
  • Enforces policies through Envoy and Envoy Gateway.
  • Supports a two-tier gateway pattern: a Tier One Gateway for authentication, top-level routing, and global rate limiting, and a Tier Two Gateway for fine-grained control over self-hosted model access with endpoint picker support.
  • Supports a wide range of AI providers including OpenAI, Azure OpenAI, Google Gemini, Vertex AI, AWS Bedrock, Mistral, Cohere, Groq, Together AI, DeepInfra, DeepSeek, Hunyuan, SambaNova, Grok, Tetrate Agent Router Service, Anthropic, BitDeer, Eden AI, Lanseq, Tencent LLM Knowledge Engine, TypeSafe AI, and Paxa Labs.
  • CRDs and API group (AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy, aigateway.envoyproxy.io) remain unchanged from the former Envoy AI Gateway.

Use cases

Application teams can use Agent Router to access multiple AI models and tools through a single, OpenAI-compatible API without managing individual provider credentials or endpoints. Platform teams can use it to enforce routing rules, rate limits, failover policies, and usage attribution across all AI services from a centralized control plane. It is suitable for both cloud-hosted and self-hosted model serving environments, including Kubernetes deployments with Envoy Gateway.

Getting started

Run Agent Router as a standalone, OpenAI-compatible router on your laptop with one command:shellOPENAI_API_KEY=sk-your-key aigw runThen point any OpenAI-compatible client at http://localhost:1975/v1. For installation and provider auto-configuration, see the CLI guide. For Kubernetes deployment with Envoy Gateway, see the Getting Started guide in the repository.

License and pricing

Agent Router is open source under the Apache 2.0 license. Pricing is not stated; the project is listed as open source.

這是什麼

Agent Router 是一個基於 Envoy Gateway 建置的開放原始碼控制平面,專為 AI 與代理流量設計。它提供統一的 OpenAI 相容 API,讓應用程式團隊能存取多種生成式 AI 模型與工具,無論是雲端託管服務、自託管推論伺服器或 MCP 伺服器。本專案前身為 Envoy AI Gateway,現為 Agentic AI Foundation 專案,採用 Apache 2.0 授權。

主要功能

  • 提供單一且一致的 OpenAI 相容 API,支援所有模型與工具。
  • 集中管理憑證、路由、配額、容錯移轉與使用歸屬,供平台團隊使用。
  • 透過 Envoy 與 Envoy Gateway 強制執行政策。
  • 支援雙層閘道模式:第一層閘道負責驗證、頂層路由與全域速率限制;第二層閘道則提供自託管模型存取的細粒度控制,並支援端點選擇器以最佳化 LLM 推論。
  • 支援多種 AI 提供者,包括 OpenAI、Azure OpenAI、Google Gemini、Vertex AI、AWS Bedrock、Mistral、Cohere、Groq、Together AI、DeepInfra、DeepSeek、Hunyuan、SambaNova、Grok、Tetrate Agent Router Service、Anthropic、BitDeer、Eden AI、Lanseq、Tencent LLM Knowledge Engine、TypeSafe AI 與 Paxa Labs。
  • CRD 與 API 群組(AIGatewayRoute、AIServiceBackend、BackendSecurityPolicy、aigateway.envoyproxy.io)與前身 Envoy AI Gateway 保持不變。

使用情境

應用程式團隊可透過單一 OpenAI 相容 API 存取多種 AI 模型與工具,無需分別管理各提供者的憑證或端點。平台團隊則可利用集中式控制平面,對所有 AI 服務強制執行路由規則、速率限制、容錯移轉政策與使用歸屬。此專案適用於雲端託管與自託管的模型服務環境,包括搭配 Envoy Gateway 的 Kubernetes 部署。

快速開始

在筆記型電腦上以單一指令執行 Agent Router 作為獨立的 OpenAI 相容路由器:shellOPENAI_API_KEY=sk-your-key aigw run然後將任何 OpenAI 相容的用戶端指向 http://localhost:1975/v1。如需安裝與提供者自動設定,請參閱 CLI 指南。如需在 Kubernetes 上搭配 Envoy Gateway 部署,請參閱儲存庫中的 Getting Started 指南。

授權與價格

Agent Router 採用 Apache 2.0 開放原始碼授權。價格未說明;該專案被歸類為開放原始碼。

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/theagentrouteragent-router.svg)](https://yunodb.com/tools/theagentrouteragent-router?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/theagentrouteragent-router?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/theagentrouteragent-router.svg" alt="Featured on YunoDB" height="20"></a>