Accessible Precision-Recall Curves in scikit-learn with py-maidr

Make scikit-learn PrecisionRecallDisplay charts accessible with py-maidr: recall and precision read by keyboard, each point against the share of positives, with the average precision and the best-F1 point.

A precision-recall curve is a classifier’s precision against its recall, one point per decision threshold. scikit-learn draws one with PrecisionRecallDisplay, and every way of building that display – from_estimator, from_predictions, from_cv_results, or a display built by hand – ends in its plot(), which is what maidr reads.

Read as a plain step line, a precision-recall curve answers the wrong questions. Whether a classifier beats guessing is how far its precision sits above the share of positives in the data – a level that is different for every dataset – and the average precision the chart is quoted by sits in the legend text, where a screen reader user is never told it. maidr reads the display instead: each point announces how far its precision sits above that baseline, and the description gives the average precision of each curve against it and the point with the best F1 score.

PR_CURVE is experimental and may change without a deprecation period: see Plot Type Stability.

Prototype

This is one of the experimental plot types. It has not been through a user study, and it may change without a deprecation period. See Plot type stability.

Setup

The examples on this page need scikit-learn, which is an optional extra: pip install maidr[sklearn]. Nothing in maidr imports it; the reading is wired up when your own code imports sklearn.metrics.

import matplotlib.pyplot as plt

# Just import maidr package: plt.show() now renders accessible output 
import maidr

One classifier

import matplotlib.pyplot as plt
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import PrecisionRecallDisplay
from sklearn.model_selection import train_test_split

import maidr  

X, y = make_classification(
    n_samples=600, n_informative=4, weights=[0.8], random_state=0
)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)
model = LogisticRegression(max_iter=500).fit(X_train, y_train)

fig, ax = plt.subplots()
# The display's own plot() is what maidr reads. The dashed chance level it
# draws is a reference, not a curve: maidr reads every point against it.
PrecisionRecallDisplay.from_estimator(  
    model, X_test, y_test, ax=ax, plot_chance_level=True  
)  
ax.set_title("Precision-recall curve of a logistic regression")

plt.show()

The curve is one series: Left and Right move along it from low recall to high, and each stop announces the recall, the precision and how far the precision sits above the share of positives – about one in five here, since the data is imbalanced. The description (d) gives the average precision, which is the number the display computed, against that baseline, and the point with the best F1 score.

Comparing classifiers

Two displays plotted on one axes are two curves of one chart, so Up and Down switch between classifiers at the same recall.

import matplotlib.pyplot as plt
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import PrecisionRecallDisplay
from sklearn.model_selection import train_test_split

import maidr  

X, y = make_classification(
    n_samples=600, n_informative=4, weights=[0.8], random_state=0
)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)

fig, ax = plt.subplots()
for model in (LogisticRegression(max_iter=500), RandomForestClassifier(random_state=0)):
    model.fit(X_train, y_train)
    PrecisionRecallDisplay.from_estimator(model, X_test, y_test, ax=ax)  
ax.set_title("Precision-recall curves of two classifiers")

plt.show()

Each curve is named the way the caller named it – the estimator’s class here, or the name= passed to the display – rather than by the legend entry, which wraps the average precision around the name. The description lists the average precision of every curve and names the highest.

The precision-recall curves maidr.keras.plot_pr_curve() and maidr.read_tensorboard_pr_curves() draw are read the same way; see the Keras and TensorBoard galleries.