semantic-router
Superfast AI decision making and intelligent processing of multi-modal data.
What is semantic-router?
Superfast AI decision making and intelligent processing of multi-modal data.
Overview of semantic-router
What it is
Semantic Router is a superfast decision-making layer designed for LLMs and agents. Instead of waiting for slow LLM generations to make tool-use decisions, it decides in milliseconds by comparing the semantic meaning of a request against predefined routes in vector space. Alternatively, it can use a small decision model in a single fast call. Every decision comes with a confidence score that can be calibrated into a real probability. The project has two release lines: the 1.x line (main branch) rebuilds the routing layer around calibrated confidence, decision models, and composable routers; the 0.x line (v0 branch) finishes at version 0.3.0 and receives only bug fixes.
Key features
- Superfast decision-making in milliseconds, avoiding slow LLM generations for tool-use decisions.
- Semantic comparison of requests against routes in vector space.
- Option to use a small decision model for a single fast call.
- Confidence calibration for each decision, enabling real probability estimates.
- Two release lines: 1.x with advanced routing features and 0.x for stable bug fixes.
Use cases
Semantic Router is used for routing user requests to appropriate LLM tools or agents. It can decide which tool or agent should handle a given query based on semantic meaning, enabling efficient multi-agent systems and tool selection. It is suitable for any application requiring fast, semantic-based decision-making for LLM interactions.
Getting started
Install the package using pip: pip install -qU 'semantic-router>=1,' For more details, see the repository README and documentation.
License and pricing
This project is open source under the MIT license.
Based on the project's public README and repository metadata.