gateway

A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.

What is gateway?

A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.

Overview of gateway

What it is

The AI Gateway is a lightweight, open-source, and enterprise-ready solution designed for fast, reliable, and secure routing to over 1,600 language, vision, audio, and image models. It provides a single, OpenAI-compatible API that allows integration with any language model in under 2 minutes. The gateway supports 45+ providers and 8+ agent frameworks, and includes integrated guardrails for input/output verification.

Key features

  • Fallbacks and automatic retries (up to 5 attempts with exponential backoff) for improved reliability
  • Load balancing across multiple API keys or providers with configurable weights
  • Multi-modal support for vision, audio (text-to-speech & speech-to-text), and image generation models using the OpenAI signature
  • 40+ pre-built guardrails for input and output verification, with support for custom guardrails
  • Secure key management with virtual keys and role-based access control (RBAC)
  • Smart caching (simple and semantic) to reduce costs and latency
  • Usage analytics for monitoring request volume, latency, costs, and error rates

Use cases

  • Routing requests to multiple LLMs and providers from a single API endpoint
  • Implementing safety and compliance guardrails on LLM inputs and outputs
  • Managing costs through caching, retries, and provider optimization
  • Integrating with agent frameworks (Autogen, CrewAI, LangChain, LlamaIndex, etc.) for complex AI workflows
  • Deploying in enterprise environments with private cloud options (AWS, Azure, GCP, Kubernetes)

Getting started

npx @portkey-ai/gateway

The gateway runs on http://localhost:8787/v1 and the console on http://localhost:8787/public/. For deployment guides (Docker, Node.js, Cloudflare, etc.) see the repository README.

License and pricing

MIT License. The project is open source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/portkey-aigateway.svg)](https://yunodb.com/tools/portkey-aigateway?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/portkey-aigateway?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/portkey-aigateway.svg" alt="Featured on YunoDB" height="20"></a>