lit

The Learning Interpretability Tool: Interactively analyze ML models to understand their behavior in an extensible and framework agnostic interface.

What is lit?

The Learning Interpretability Tool: Interactively analyze ML models to understand their behavior in an extensible and framework agnostic interface.

Overview of lit

What it is

The Learning Interpretability Tool (LIT), formerly known as the Language Interpretability Tool, is a visual, interactive ML model-understanding tool. It supports text, image, and tabular data. LIT can be run as a standalone server or inside notebook environments such as Colab, Jupyter, and Google Cloud Vertex AI notebooks. It is designed to help answer questions about model behavior, such as which examples a model performs poorly on, why a model made a particular prediction, and whether the model behaves consistently under changes like textual style, verb tense, or pronoun gender.

Key features

  • Local explanations via salience maps and rich visualization of model predictions.
  • Aggregate analysis including custom metrics, slicing and binning, and visualization of embedding spaces.
  • Counterfactual generation through manual edits or generator plug-ins to dynamically create and evaluate new examples.
  • Side-by-side mode to compare two or more models, or one model on a pair of examples.
  • Highly extensible to new model types, including classification, regression, span labeling, seq2seq, and language modeling. Supports multi-head models and multiple input features out of the box.
  • Framework-agnostic and compatible with TensorFlow, PyTorch, and more.

Use cases

LIT is built for debugging model performance, understanding individual predictions, and testing model consistency. It can be used to identify failure modes, attribute predictions to input features, and explore counterfactual scenarios. The tool supports workflows for classification, regression, and generative AI tasks, and allows side-by-side comparison of different models or data points.

Getting started

LIT can be installed via pip: sh pip install lit-nlp To install dependencies for specific examples: sh pip install 'lit-nlp[examples-discriminative-ai]' pip install 'lit-nlp[examples-generative-ai]' For a quick-start with classification and regression models from the GLUE benchmark: sh python -m lit_nlp.examples.glue.demo --port=5432 --quickstart Then navigate to http://localhost:5432 to access the LIT UI. Notebook usage is also supported; see the Colab demos in the repository.

License and pricing

LIT is open source under the Apache-2.0 license. The pricing type is open_source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/pair-codelit.svg)](https://yunodb.com/tools/pair-codelit?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/pair-codelit?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/pair-codelit.svg" alt="Featured on YunoDB" height="20"></a>