cascadeflow
Cascading runtime for AI agents. Optimize cost, latency, quality, and policy decisions inside the agent loop.
What is cascadeflow?
Cascading runtime for AI agents. Optimize cost, latency, quality, and policy decisions inside the agent loop.
Overview of cascadeflow
What it is
cascadeflow is an in-process intelligence layer for AI agents. It optimizes cost, latency, quality, budget, compliance, and energy inside the execution loop, not at the HTTP boundary. It works with LangChain, OpenAI Agents SDK, CrewAI, PydanticAI, Google ADK, n8n, Vercel AI SDK, and Hermes Agent. The overhead is sub-5ms.
Key features
- Speculative execution with quality validation: small, fast models are tried first, then escalated to larger models only when quality validation fails.
- Dynamic escalation based on configurable thresholds (completeness, confidence, correctness).
- Learns patterns to optimize future cascading decisions and domain-specific routing.
- Zero configuration; works with existing models from over 17 providers.
- Three tiers of integration: zero-change observability, scoped runs with budget, and decorated agents with policy.
- Hermes Agent integration for per-skill model cascading, task-complexity cascading, topic-aware subagent cascading, observe-mode rollout, and auditable decisions.
Use cases
- Reducing cost and latency while maintaining quality in AI agent loops.
- Enforcing budget, compliance, and energy constraints per step or per tool call.
- Making per-step model decisions based on agent state, tool results, and quality scores.
- Injecting business KPI weights (quality, cost, latency) during agent execution.
- Running agent cascades with Hermes Agent for skill-specific model selection.
Getting started
bash pip install cascadeflow bash npm install @cascadeflow/core python from cascadeflow import CascadeAgent, ModelConfig agent = CascadeAgent(models=[ ModelConfig(name='nous/hermes-flash', provider='openai', cost=0.000375), ModelConfig(name='gpt-5', provider='openai', cost=0.00562), ]) result = await agent.run('What is the capital of France?') print(f'Answer: {result.content}') print(f'Model used: {result.model_used}') print(f'Cost: ${result.total_cost:.6f}')
License and pricing
The license is MIT. The project is open source with no stated pricing.
Based on the project's public README and repository metadata.