awesome-conformal-prediction

Overview of awesome-conformal-prediction

What it is

This repository is a curated list of resources about conformal prediction (also called conformal inference). Conformal prediction is a distribution-free, model-agnostic framework for quantifying predictive uncertainty. Instead of a single point prediction, it returns a prediction set (for classification) or a prediction interval (for regression) guaranteed to contain the true label with a user-chosen probability (e.g., 90% or 95%) under the assumption that the data are exchangeable. These guarantees hold in finite samples, for any underlying model, and without distributional assumptions. The list is maintained by Dr. Valeriy Manokhin, who completed his PhD under Prof. Vladimir Vovk, the creator of conformal prediction.

Key features

• Covers topics: conformal prediction, conformal inference, prediction intervals, uncertainty quantification, model calibration, distribution-free inference, selective prediction, trustworthy machine learning, LLM uncertainty, time series forecasting, anomaly detection.
• Includes sections for tutorials, courses, books, articles, videos and talks, presentation slides.
• Lists open-source libraries and tools for Python, R, Julia, and other languages, plus AI platforms and Kaggle notebooks.
• Contains research papers (general, time series forecasting, anomaly detection) and PhD theses.
• Provides community resources: events, workshops, conferences, researchers, websites, Twitter/X.
• Includes a quickstart code example using the crepes library with scikit-learn.

Use cases

Conformal prediction can be applied to regression, classification, time series, anomaly detection, and LLM uncertainty. It is used for uncertainty quantification, trustworthy machine learning, selective prediction, and safe deployment of AI systems. The repository is intended for researchers, practitioners, and production teams.

Getting started

pythonimport numpy as npfrom crepes import ConformalRegressorfrom sklearn.ensemble import RandomForestRegressorfrom sklearn.datasets import fetch_california_housingfrom sklearn.model_selection import train_test_splitX, y = fetch_california_housing(return_X_y=True)X_train, X_rest, y_train, y_rest = train_test_split(X, y, test_size=0.4, random_state=0)X_cal, X_test, y_cal, y_test = train_test_split(X_rest, y_rest, test_size=0.5, random_state=0)model = RandomForestRegressor(n_estimators=200, random_state=0).fit(X_train, y_train)residuals_cal = np.abs(y_cal - model.predict(X_cal))cr = ConformalRegressor().fit(residuals=residuals_cal)intervals = cr.predict(y_hat=model.predict(X_test), confidence=0.9) # 90% intervalsRunnable notebooks live in examples/. See CONTRIBUTING.md to add resources.

License and pricing

The repository is licensed under CC BY-NC-ND 4.0. Academic work must cite. The pricing type is open source.

Based on the project's public README and repository metadata.