maidr reads the training curves of a TensorBoard log directory. Pass the directory you would start TensorBoard with to maidr.read_tensorboard_scalars(), and hand any chart it returns to maidr.show(), maidr.render() or maidr.save_html(): the curve can then be read from the keyboard, as sound, as text and in braille. Neither TensorFlow nor TensorBoard needs to be installed. For charts drawn in Python, see the Matplotlib / Seaborn and other gallery pages.
TensorBoard support is experimental. What it reads, and how, may change without a deprecation period: see Plot Type Stability.
Each scalar tag, such as epoch_loss, becomes one line chart, with the step along the x axis and one line per run, as on TensorBoard’s Scalars dashboard. A run is a directory under the log directory holding event files: Keras’s TensorBoard callback writes a train and a validation run, so the two curves of a model can be compared point by point.
| Written by | read |
|---|---|
Keras keras.callbacks.TensorBoard |
yes |
tf.summary.scalar, TensorFlow 1 and 2 |
yes |
PyTorch torch.utils.tensorboard.SummaryWriter, tensorboardX |
yes |
| Distributions | yes |
| Histograms, as ridgelines [experimental] | yes |
| Hyperparameter sweeps (HParams), as a scatter matrix and as parallel coordinates [experimental] | yes |
| Embedding Projector, as a 2D scatter of the principal components | yes |
| PR curves | yes |
Profile: step-time graph and top operations (needs xprof) |
yes |
| The Keras model graph, as a directed graph [experimental] | yes |
| Images, op-level graphs | not yet |
The examples read tensorboard-logs/, the log directory of a small model trained for twelve epochs with the TensorBoard callback.
Install the tensorboard extra and TensorBoard shows a maidr tab beside its own dashboards:
The tab lists every chart py-maidr reads from the log directory – scalars, distributions, histograms, PR curves, a hyperparameter sweep, the Embedding Projector, the Keras model graph and, with xprof installed, a profile – in one labelled picker, and shows the one picked, read by maidr. The Reload button reads the log directory again, so a run that is still training can be followed; nothing reloads on its own, so your place in a chart is never taken from you. The page works without a network: maidr.js is served with it.
maidr.read_tensorboard_scalars() returns one chart per tag, each with its tag and the runs drawn on it.
epoch_accuracy | ('train', 'validation')
epoch_learning_rate | ('train',)
epoch_loss | ('train', 'validation')
evaluation_accuracy_vs_iterations | ('validation',)
evaluation_loss_vs_iterations | ('validation',)Save every tag as its own page with a loop, or only the ones you name with tags=:
To follow a run while it trains, read the directory again: each call reads the event files as they are at that moment.
Smoothing is on by default, at TensorBoard’s own weight of 0.6, and each run is drawn twice: as logged, and smoothed under its name followed by (smoothed). The left and right arrow keys move along the epochs; up and down move to the line above or below at that epoch, whichever run it is. Where two lines meet, the point is announced as their intersection.
smoothing=0 draws only the values as logged, one line per run.
A long run is thinned to 1,000 points per line, evenly spaced and keeping the first and last step, so it stays quick to walk; max_points=None keeps every point. A loss that became NaN is a gap in the line, never a zero.
maidr.read_tensorboard_distributions() reads what TensorBoard’s Distributions dashboard shows: for each step, the minimum, the median, the maximum and the six points a normal distribution puts at one, two and three standard deviations from the middle. They are drawn as TensorBoard draws them, nested bands around the median, and read as nine lines named for the share of values below each, such as 84.1%. At a step, up and down move through them in value order, which is how wide the distribution is there; left and right follow one of them through training.
maidr.read_tensorboard_pr_curves() reads what TensorBoard’s PR Curves dashboard shows: per tag, each run’s precision against its recall, one point per decision threshold, at the run’s last step (or step=). Each line is named with its average precision and the precision a classifier guessing at random reaches, the share of positives, so you hear at once whether a run is any better than chance: good (AP 0.99, chance 0.32). The chance level is drawn as a dashed line. A threshold at which nothing was predicted positive has no precision and is left out. The example reads tensorboard-pr-curves/, written by PyTorch’s add_pr_curve for a good and a poor classifier.
maidr.read_tensorboard_profile() reads the two charts TensorBoard’s Profile dashboard leads with, from a profile captured with tf.profiler or the Keras TensorBoard callback’s profile_batch: the step-time graph, one stacked bar per step split into where its time went – host compute, device compute, input, compilation and the rest – and the top operations, the kinds of operation that took the most self time, largest first. Profiles are stored as XPlane files that only the profile plugin’s converter reads, so this one reader needs pip install xprof, as TensorBoard’s own Profile tab does.
The converter’s tables can also be drawn directly with maidr.tensorboard.profile.profile_charts(). The example draws tensorboard-profile/, the tables of a profile of eight training steps of a small Keras model.
from pathlib import Path
from maidr.tensorboard.profile import profile_charts
step_time, top_operations = profile_charts(
Path("tensorboard-profile/input_pipeline_analyzer.json").read_text(),
Path("tensorboard-profile/framework_op_stats.json").read_text(),
)
maidr.show(step_time)
maidr.show(top_operations)maidr.read_tensorboard_hparams() reads a sweep logged with the HParams plugin’s hp.hparams_config and hp.hparams, as the HParams dashboard shows it, and returns two charts. The second is its scatter plot matrix: a row per metric and a column per hyperparameter, one point per training session. Page Up and Page Down move between the cells. The example reads tensorboard-hparams/, four sessions of a sweep over the learning rate, optimizer, units and dropout, measuring accuracy and validation loss.
maidr.read_tensorboard_projector() reads the embeddings TensorBoard’s Embedding Projector shows, from the log directory’s projector_config.pbtxt, and draws each in two dimensions: its first two principal components, each axis named with the share of the variance it explains. Each label from the metadata is a layer of its own, so Page Up and Page Down move between classes and you can hear which sit apart and which overlap. Pass coordinates= to draw a t-SNE or UMAP projection computed elsewhere instead. The example reads tensorboard-projector/, written by PyTorch’s add_embedding: three labelled clusters of eight-dimensional vectors.
A checkpoint written by Keras or TensorFlow is read only with TensorFlow installed; PyTorch writes its vectors as text, which needs nothing.
maidr.read_tensorboard_histograms() reads what TensorBoard’s Histograms dashboard shows: one chart per tag and run, with a ridge for each logged step, the earliest at the bottom. Left and right move along one step’s values; up moves to a later step and down to an earlier one, holding the value, so you hear how the count at that value changes over training. Every step is counted in the same 30 bins, and at most 51 steps are drawn, as TensorBoard draws at most 51; bins= and max_steps= change both.
A ridge’s height is its count against the tallest count of any step, as in TensorBoard, so one step much taller than the rest flattens the others; its count is still read out exactly. These ridgelines are an experimental plot type. The example reads tensorboard-histograms/, a distribution logged with tf.summary.histogram every four steps, whose center drifts right and whose spread widens.
The first chart maidr.read_tensorboard_hparams() returns is the sweep’s parallel coordinates: one line per session across an axis for each hyperparameter and then each metric. Right and left move along one session’s axes, each value read in its own units; up and down move to another session. A hyperparameter that is a word or a yes-or-no is placed by its position among the values tried, and its axis says which is which, such as optimizer (0 adam, 1 sgd). Parallel coordinates are an experimental plot type.
maidr.read_tensorboard_graph() reads what TensorBoard’s Graphs dashboard shows under its Keras tag: the model Keras’s TensorBoard callback logs with write_graph=True, its default. One chart per run that logged a model, drawn as maidr.keras.plot_model() draws it: one box per layer, top to bottom from the inputs, with an arrow from each layer to the layers called on its output. Left and right walk the layers in the order data flows through them; the Inputs and Outputs rotor units follow the arrows. expand_nested=True opens a model used as a layer into a scope of its own layers. The example reads tensorboard-graph/, a small functional model with a residual block and a skip connection around it, trained for one epoch.
Only Keras’s own record of the model is read, not the op-level graph TensorFlow can log beside it, which lists every operation and runs to hundreds of nodes for a few layers. A directed graph is an experimental plot type, and reading one needs a maidr.js release that carries the directed_graph trace.