evidently

Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen A

What is evidently?

Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

Overview of evidently

What it is

Evidently is an open-source Python library for evaluating, testing, and monitoring machine learning and large language model systems. It works with tabular and text data, supporting both predictive and generative tasks such as classification, regression, and RAG. The framework includes over 100 built-in metrics, from data drift detection to LLM judges, and allows custom metrics via a Python interface. It supports offline evaluations and live monitoring, with an open architecture for data export and integration with existing tools.

Key features

  • Reports and Test Suites: Compute and summarize data, ML, and LLM quality evaluations. Reports can be viewed interactively in Python or exported as JSON, HTML, or Python dictionary. Test Suites add pass/fail conditions for regression testing and CI/CD checks.
  • Monitoring Dashboard: A self-hosted or cloud-based UI to visualize metrics and test results over time. The open-source version can be self-hosted; Evidently Cloud offers a managed version with additional features.
  • 100+ built-in metrics covering text descriptors, LLM outputs, data quality, data distribution drift, classification, regression, ranking (including RAG), and recommendations.
  • Custom metrics: Python interface to define custom evaluators.
  • Open architecture: Easy data export and integration with existing tools.

Use cases

  • Experiment evaluation and exploratory analysis - Debugging ML and LLM systems - Regression testing and CI/CD pipeline checks - Data validation and quality monitoring - Live monitoring of production models and data pipelines

Getting started

Install Evidently via pip or conda: sh pip install evidently or sh conda install -c conda-forge evidently To launch the monitoring dashboard with demo projects: sh evidently ui --demo-projects all For detailed tutorials and examples, see the repository README.

License and pricing

Evidently is open source under the Apache-2.0 license. The framework itself is free to use. Evidently Cloud is a separate managed service that offers a free tier and additional features such as dataset management, alerting, and no-code evaluations.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/evidentlyaievidently.svg)](https://yunodb.com/tools/evidentlyaievidently?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/evidentlyaievidently?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/evidentlyaievidently.svg" alt="Featured on YunoDB" height="20"></a>