Interpretation of Predictions

Classes to build objects to better interpret predictions of a model

Interpretation

def Interpretation(
    learn:Learner, dl:DataLoader, # `DataLoader` to run inference over
    losses:TensorBase, # Losses calculated from `dl`
    act:NoneType=None, # Activation function for prediction
):

Interpretation base class, can be inherited for task specific Interpretation classes

Interpretation is a helper base class for exploring predictions from trained models. It can be inherited for task specific interpretation classes, such as ClassificationInterpretation. Interpretation is memory efficient and should be able to process any sized dataset, provided the hardware could train the same model.

Note

Interpretation is memory efficient due to generating inputs, predictions, targets, decoded outputs, and losses for each item on the fly, using batch processing where possible.

source

Interpretation.from_learner

def from_learner(
    learn, # Model used to create interpretation
    ds_idx:int=1, # Index of `learn.dls` when `dl` is None
    dl:DataLoader=None, # `Dataloader` used to make predictions
    act:NoneType=None, # Override default or set prediction activation function
):

Construct interpretation object from a learner

source

Interpretation.top_losses

def top_losses(
    k:int | None=None, # Return `k` losses, defaults to all
    largest:bool=True, # Sort losses by largest or smallest
    items:bool=False, # Whether to return input items
):

k largest(/smallest) losses and indexes, defaulting to all losses.

With the default of k=None, top_losses will return the entire dataset’s losses. top_losses can optionally include the input items for each loss, which is usually a file path or Pandas DataFrame.

source

Interpretation.plot_top_losses

def plot_top_losses(
    k:int | collections.abc.MutableSequence, # Number of losses to plot
    largest:bool=True, # Sort losses by largest or smallest
    **kwargs
):

Show k largest(/smallest) preds and losses. Implementation based on type dispatch

To plot the first 9 top losses:

interp = Interpretation.from_learner(learn)
interp.plot_top_losses(9)

Then to plot the 7th through 16th top losses:

interp.plot_top_losses(range(7,16))

source

Interpretation.show_results

def show_results(
    idxs:list, # Indices of predictions and targets
    **kwargs
):

Show predictions and targets of idxs

Like Learner.show_results, except can pass desired index or indicies for item(s) to show results from.

source

ClassificationInterpretation

def ClassificationInterpretation(
    learn:Learner, dl:DataLoader, # `DataLoader` to run inference over
    losses:TensorBase, # Losses calculated from `dl`
    act:NoneType=None, # Activation function for prediction
):

Interpretation methods for classification models.

source

ClassificationInterpretation.confusion_matrix

def confusion_matrix():

Confusion matrix as an np.ndarray.

source

ClassificationInterpretation.plot_confusion_matrix

def plot_confusion_matrix(
    normalize:bool=False, # Whether to normalize occurrences
    title:str='Confusion matrix', # Title of plot
    cmap:str='Blues', # Colormap from matplotlib
    norm_dec:int=2, # Decimal places for normalized occurrences
    plot_txt:bool=True, # Display occurrence in matrix
    **kwargs
):

Plot the confusion matrix, with title and using cmap.

source

ClassificationInterpretation.most_confused

def most_confused(
    min_val:int=1
):

Sorted descending largest non-diagonal entries of confusion matrix (actual, predicted, # occurrences

source

SegmentationInterpretation

def SegmentationInterpretation(
    learn:Learner, dl:DataLoader, # `DataLoader` to run inference over
    losses:TensorBase, # Losses calculated from `dl`
    act:NoneType=None, # Activation function for prediction
):

Interpretation methods for segmentation models.