openllmetry
Open-source observability for your GenAI or LLM application, based on OpenTelemetry
What is openllmetry?
Open-source observability for your GenAI or LLM application, based on OpenTelemetry
Overview of openllmetry
What it is
OpenLLMetry is an open-source observability tool for GenAI and LLM applications, built on top of OpenTelemetry. It provides standard OpenTelemetry instrumentations for LLM providers, vector databases, and frameworks. The project is maintained by Traceloop under the Apache 2.0 license.
Key features
- Built on OpenTelemetry, allowing connection to existing observability solutions such as Datadog, Honeycomb, and others.
- Provides instrumentations for LLM providers (e.g., OpenAI, Anthropic, Bedrock, Cohere, Gemini, Mistral, Ollama) and vector databases (e.g., Chroma, Pinecone, Qdrant, Weaviate).
- Supports frameworks including LangChain, LlamaIndex, Haystack, CrewAI, LangGraph, and LiteLLM.
- Includes a Traceloop SDK that simplifies setup while outputting standard OpenTelemetry data.
- Can be used with multiple destinations: Traceloop, Axiom, Azure Application Insights, Datadog, Dynatrace, Google Cloud, Grafana, Honeycomb, New Relic, OpenTelemetry Collector, and more.
- No telemetry is collected in the SDK or instrumentations (as of version v0.49.2 and above).
Use cases
- Monitoring and debugging LLM applications in production.
- Integrating LLM observability into existing OpenTelemetry-based monitoring stacks.
- Tracing calls to multiple LLM providers and vector databases from a single instrumentation layer.
- Observing framework-level operations in LangChain, LlamaIndex, or similar tools.
Getting started
Install the SDK:bashpip install traceloop-sdkThen add to your code:pythonfrom traceloop.sdk import TraceloopTraceloop.init()For local testing, disable batch sending:pythonTraceloop.init(disable_batch=True)
License and pricing
OpenLLMetry is open-source under the Apache 2.0 license. Pricing is not stated; the project is listed as open source.
Based on the project's public README and repository metadata.