semantic-router

An open, programmable decision layer for models and compute.

What is semantic-router?

An open, programmable decision layer for models and compute.

Overview of semantic-router

What it is

semantic-router is an open, programmable decision layer for models and compute. It provides a single API that selects or combines models for each call, guided by user-defined policy. The router works across local, private, and cloud compute backends, allowing agent harnesses to maintain their own agent loop, tools, and task state while the router handles model selection and backend routing.

Key features

  • Selects or combines models per call based on policy.
  • Supports multiple backends across local, private, and cloud compute.
  • Allows setting priorities for quality, latency, and cost.
  • Keeps calls within approved geographic locations.
  • Provides a single API for many models.
  • Integrates with existing agent harnesses without replacing them.

Use cases

semantic-router is used in scenarios where different models excel at different tasks, hardware varies in speed and capacity, and compute must be distributed across edge, private, and cloud locations. It is suitable for production LLM inference systems that need dynamic model selection, mixture-of-models architectures, and policy-driven routing. Example applications include real-time hallucination detection, cursor-style auto model selection, and category-aware semantic caching.

Getting started

bashcurl -fsSL https://vllm-sr.ai/install.sh | bash -s -- --channel stableFor pip, uv, or agent-driven installation, see the repository README.

License and pricing

This project is licensed under Apache-2.0 and is open source. Pricing is not stated.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/vllm-projectsemantic-router.svg)](https://yunodb.com/tools/vllm-projectsemantic-router?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/vllm-projectsemantic-router?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/vllm-projectsemantic-router.svg" alt="Featured on YunoDB" height="20"></a>