future-agi
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Gu
What is future-agi?
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
Overview of future-agi
What it is
Future AGI is an open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. It provides tracing, evals, simulations, datasets, gateway, and guardrails in a single platform. The project is self-hostable and licensed under Apache 2.0.
Key features
- Simulate: Generate thousands of multi-turn conversations with realistic personas, adversarial inputs, and edge cases, including text and voice (LiveKit, VAPI, Retell, Pipecat).
- Evaluate: 50+ metrics under one
evaluate()call, covering groundedness, hallucination, tool-use correctness, PII, tone, and custom rubrics, using LLM-as-judge, heuristic, and ML methods. - Protect: 18 built-in scanners (PII, jailbreak, injection, etc.) plus 15 vendor adapters (Lakera, Presidio, Llama Guard, etc.), inline in gateway or as standalone SDK.
- Monitor: OpenTelemetry-native tracing across 50+ frameworks (LangChain, LlamaIndex, CrewAI, DSPy, etc.) with span graphs, latency, token cost, and live dashboards.
- Agent Command Center: OpenAI-compatible gateway supporting 100+ providers, 15 routing strategies, semantic caching, virtual keys, MCP, and A2A. Performance: ~29k req/s, P99 ≤ 21ms with guardrails on.
- Optimize: Six prompt-optimization algorithms (GEPA, PromptWizard, ProTeGi, Bayesian, Meta-Prompt, Random) that use production traces as training data.
Use cases
- Shipping self-improving AI agents that simulate edge cases before launch, evaluate production behavior, protect users in real time, and turn every trace into signal for the next version.
- Replacing a stack of multiple tools (Langfuse, Braintrust, Helicone, Guardrails AI, custom simulator) with a single platform covering the lifecycle: simulate → evaluate → protect → monitor → optimize.
- Self-hosting for data sovereignty or using the managed Cloud; dropping in existing stack via OTel or OpenAI-compatible HTTP.
Getting started
bash git clone https://github.com/future-agi/future-agi.git cd future-agi ./bin/install # Windows (PowerShell): ./.bin/install.ps1 After installation, open http://localhost:3000, create an account, copy API key and secret key from the sidebar, then send your first trace using the provided Python or TypeScript instrumentation.
License and pricing
Apache 2.0 open-source license. The project is free and self-hostable; a managed Cloud version is also available with a free tier (no credit card required).
Based on the project's public README and repository metadata.