Verified project record

jessevig/bertviz

bertviz is a library that provides interactive visualizations of attention weights in Transformer language models directly within Jupyter or Colab notebooks. It offers multiple views—including head, model, and neuron perspectives—to help users explore how attention mechanisms operate.

Project overview

By turning opaque attention weights into interactive, multi-perspective visualizations inside familiar notebook environments, bertviz fills a practical gap for researchers and engineers who need to inspect Transformer model internals without building custom tooling.

Project type: Model Development · Model Runtime · Data Processing
Deployment: Refer to project documentation
License: Apache-2.0

Best for

Researchers, developers, and AI engineers who need to understand and interpret attention mechanisms in Transformer language models.

Key capabilities

Visualizes attention for one or more attention heads within the same layer.
Shows a bird's-eye view of attention across all layers and heads.
Visualizes individual neurons in the query and key vectors and shows how they are used to compute attention.
Supports visualizing attention for encoder-decoder models such as BART and T5.
Allows retrieving generated HTML representations to save as independent files or use with custom display methods.
Provides a drop-down menu to filter attention based on which sentence the tokens belong to.

Limitations and risks

Designed for shorter inputs and may run slowly if the input text is very long or the model is very large.
Neuron view only supports the custom BERT, GPT-2, and RoBERTa models included with the tool.
Visualizing attention weights does not necessarily provide a direct explanation for model predictions.
On Google Colab, some visualizations will fail due to runtime disconnection when the input text is long.

Getting started

Install the library and its notebook dependencies by running: pip install bertviz, then pip install jupyterlab ipywidgets, and start the environment with jupyter lab.

Evidence and sources

README: BertViz is an interactive tool for visualizing attention in Transformer language models. It can be run inside a Jupyter or Colab notebook through a simple Python API
README: Visualizing attention weights illuminates one type of architecture within the model but does not necessarily provide a direct *explanation* for predictions
README: A Multiscale Visualization of Attention in the Transformer Model
README: The *head view* visualizes attention for one or more attention heads in the same layer.
README: The *model view* shows a bird's-eye view of attention across all layers and heads.

Reviewed at: 2026-07-16T12:16:25Z