#  SI-MIL: Taming Deep MIL for Self-Interpretability in Gigapixel
Histopathology

Saarthak Kapse1 , Pushpak Pati211footnotemark: 1 , Srijan Das3, Jingwei
Zhang1, Chao Chen1, Maria Vakalopoulou4,
Joel Saltz1, Dimitris Samaras1, Rajarsi R. Gupta1, Prateek Prasanna1

1Stony Brook University, USA
2Johnson & Johnson, Switzerland
3UNC Charlotte, USA
4CentraleSupélec, University of Paris-Saclay, France These authors contributed
equally to this paper.

###### Abstract

Introducing interpretability and reasoning into Multiple Instance Learning
(MIL) methods for Whole Slide Image (WSI) analysis is challenging, given the
complexity of gigapixel slides. Traditionally, MIL interpretability is limited
to identifying salient regions deemed pertinent for downstream tasks, offering
little insight to the end-user (pathologist) regarding the rationale behind
these selections. To address this, we propose Self-Interpretable MIL (SI-MIL),
a method intrinsically designed for interpretability from the very outset. SI-
MIL employs a deep MIL framework to guide an interpretable branch grounded on
handcrafted pathological features, facilitating linear predictions. Beyond
identifying salient regions, SI-MIL uniquely provides feature-level
interpretations rooted in pathological insights for WSIs. Notably, SI-MIL,
with its linear prediction constraints, challenges the prevalent myth of an
inevitable trade-off between model interpretability and performance,
demonstrating competitive results compared to state-of-the-art methods on WSI-
level prediction tasks across three cancer types. In addition, we thoroughly
benchmark the local- and global-interpretability of SI-MIL in terms of
statistical analysis, a domain expert study, and desiderata of
interpretability, namely, user-friendliness and faithfulness.

†† Code: https://github.com/bmi-imaginelab/SI-MIL/tree/main

##  1 Introduction

In the last decade, advancements in deep learning techniques, especially
Multiple Instance Learning (MIL) algorithms [21, 36, 56], have dramatically
revolutionized computational pathology, which has transitioned from analyzing
local regions-of-interest [39] to gigapixel whole slide images (WSIs). A
standard MIL workflow takes in feature representations of patches from a WSI,
embedded via a deep neural network, and aggregates them to define a slide-
level representation adept for a downstream task. While these deep neural
network-reliant workflows have resulted in high performance, they often lack
pathologist-friendly interpretability and reasoning in their predictions [52],
which is crucial for building trust in routine clinical workflows and defining
reliability and accountability of AI algorithms [54, 7, 18], particularly in
clinical contexts.

Figure 1:  Unlike conventional MIL, SI-MIL co-learns from deep and
pathological features. While both MILs offer patch-level interpretability,
only ours provides pathological feature-level rationale for WSI predictions.
The attention maps in SI-MIL are grounded on geometrically and physically-
interpretable descriptors.

In computational pathology, efforts to achieve WSI-level interpretability have
predominantly focused on two directions: (1) identifying salient regions in a
WSI, and (2) employing post-hoc techniques to elucidate the underlying
patterns in salient regions. The first approach, employed by traditional MIL,
includes techniques such as visualization of attention maps [21, 36, 56, 68,
57, 67] and post-hoc gradient-based saliency [70, 47], which highlight image
patches that influence the model prediction. These techniques, though useful,
may not offer a complete understanding of the model’s decisions and can result
in visualizations that are hard for experts to interpret due to a lack of
user-friendly feature grounding [4]. The latter approach involves extracting
interpretable pathological features from the MIL-identified salient patches
and then conducting statistical analyses to find correlations between these
features and the WSI ground truths in a post-hoc manner [43, 10, 15]. However,
there is a clear disconnect between the deep features used for MIL training
and the pathological features. Using post-hoc pathological features to explain
deep models can be sub-optimal [51]. Furthermore, patches with high attention
may be crucial for deep feature space, but may not be optimal in pathological
feature space, thus compromising interpretability.

To truly interpret a prediction model, it seems inevitable to bring
interpretable features into training. A natural idea is to directly train MILs
using these features, followed by statistical analysis of features from highly
attended patches. However, this will not exploit the full potential of deep
learning, as our analysis in Section 4 will reveal. This brings us to the main
question: _Can we really achieve inherent interpretability without
compromising model performance?_

The answer is yes. In this paper, we provide the first WSI solution with both
inherent interpretability and strong prediction power. Our key observation is
that a highly accurate deep model is not unique; there can be many optimal or
close-to-optimal deep models for a dataset/task, due to over-parameterization
[45, 27, 30]. Therefore, we hypothesize that we can alter the learning
procedure and find an alternative model with desired interpretability and
still be powerful in prediction. In particular, we propose to pair a deep MIL
model with an interpretable model grounded on pathological features during
training. Through co-learning, the MIL retains its predictive power.
Meanwhile, it is sufficiently “tamed” by the co-learned interpretable model,
which renders interpretability. As shown in Fig. 1, the tamed deep MIL model
has a different attention map from the standard MIL. It is attending to
patches which can also be discriminated by the companion interpretable model.

Our method, Self-Interpretable MIL (SI-MIL), is a dual branch network,
consisting of a conventional MIL and a novel Self-Interpretable (SI) branch.
The MIL exploits deep features’ discriminative power to guide the SI branch.
Grounded on pathological features, the SI branch then provides a linear
prediction. A differentiable Top-$K$ operator for selecting patches, connects
the two branches and enables end-to-end co-learning. To highlight, SI-MIL is
inherently interpretable [51, 5] due to the linear mapping between the
pathological features and the output predictions. Therefore, it can reflect
the impact of each feature on the output, thus providing a feature-level
rationale, as shown in Fig. 1. Also, by leveraging the potential of a deep
feature extractor, state-of-the-art MIL, and geometrically and physically-
interpretable pathological features, SI-MIL counters the well-known myth of
unavoidable model interpretability and performance trade-off [4, 51]. Notably,
SI-MIL is generic enough to substitute any state-of-the-art MIL method in the
MIL branch. In summary, our main contributions are:

  * •

SI-MIL, the first interpretable-by-design MIL method for gigapixel WSIs, which
provides de novo feature-level interpretations grounded on pathological
insights for a WSI.

  * •

A novel co-learning strategy for SI-MIL to mitigate the model performance-
interpretability trade-off associated with self-interpretable methods. We
quantitatively establish the efficacy of our method for classification tasks
on three cancer types.

  * •

We demonstrate the utility of SI-MIL’s local WSI-level and global cohort-level
explanations thorough quantitative and qualitative benchmarking in terms of
statistical analysis, a domain expert study, and desiderata of
interpretability, _i.e_., fidelity, user-friendliness and faithfulness.

  * •

We provide a comprehensive dataset for $\sim$2.2K WSIs, featuring nuclei maps,
pathological features, and SI-MIL derived outputs, with the aim of
streamlining the resource intensive preprocessing towards interpretability
studies in computational pathology.

##  2 Related work

This section presents an overview of different forms of interpretability,
primarily focusing on the domain of computational pathology.

Post-hoc interpretability methods: These methods fall into two categories:
patch-level and WSI-level. Patch-level techniques, like GradCAM and Layer-Wise
Relevance Propagation (used in [19, 53]), highlight key pixels in model
predictions. For deeper insight, studies like [22, 23] use biological entity-
based graphs for pathologist-friendly explanations. At the WSI-level,
interpretability is primarily achieved through attention maps that identify
salient regions in WSIs. Additionally, few methods, such as those by [70, 47],
use segmentation maps or gradient-based techniques to localize significant
areas. However, these methods, as [51] notes, may suffer from a disconnect
from the model’s computations. In pathology, this is particularly evident when
comparing the deep features used for MIL training and the pathological
features used for subsequent analysis [43, 10, 15], revealing a disparity in
the features for training versus those for feature correlation.

Vision-Language methods: Previous works [48, 37, 20, 32] have explored
interpretability using task reasoning through textual descriptions or vision-
language similarity [2, 38, 69]. However these methods [38, 20] either suffer
from post-hoc approximation [51], or are not scalable [69, 32] for gigapixel
images. Note that in pathology, most paired image-text data are only at the
patch level [20]. This makes it challenging to design WSI-level interpretable
prediction models from only patch-level descriptions. Furthermore, at WSI-
level, unlike natural images, the text descriptions in pathology reports are
not holistic; i.e. these reports do not capture the complete landscape and
primarily consist of global summaries of the pathologists’ findings.

Self-interpretability methods: A family of models, grounded in concepts [29,
14, 46, 65, 64], has become prominent for natural image interpretation. These
models learn interpretable embeddings by mapping visual representations to a
concept layer, and linearly aggregate these for prediction. Challenges such as
information leakage and semantic inaccuracies are noted in [41, 40, 5]. To
address this, [5] uses concept embeddings [14] to learn syntactic rules and
make predictions based on concept truths. While effective in interpretability,
its validation is confined to Boolean logic tasks. Despite its emergence in
the analysis of natural images, this field has yet to be explored in the
context of gigapixel pathology. Building upon this, our proposed SI-MIL can
handle complex WSI tasks while embedding interpretability directly into the
MIL framework.

##  3 Method

Figure 2: Overview of SI-MIL: Conventional MIL branch guides the Patch
Attention-Guided Top-$K$ (PAG Top-K) patch selection module to select the
pathological features of key regions from WSI, followed by linear scaling in
the Self-Interpretable branch, and linear prediction.

In this section, we present the details of our dual branch SI-MIL (overview in
Fig. 2), consisting of a conventional MIL branch and a Self-Interpretable (SI)
branch, for analyzing WSIs. We describe the conventional MIL in Sec. 3.1 and
present the complete SI-MIL framework in Sec. 3.2. Detailed description of the
feature extraction pipelines, _i.e_., the process of extracting black-box deep
features ($g$) and interpretable pathological features ($f$), is provided in
Sec. 3.3.

###  3.1 Conventional MIL

In conventional MIL, each WSI is decomposed into patches ($p_{1}$, $p_{2}$,
…$p_{N}$), and their extracted features ($g_{1}$, $g_{2}$, …$g_{N}$),
$g_{i}\in\mathbb{R}^{D}$ are treated as a bag of instances. In this work, we
leverage an additive version of MIL [24] in the conventional MIL, which
imparts better spatial credit assignment to tissue regions in a WSI. As
illustrated in Fig. 2b, conventional MIL consists of a projector $H(\cdot)$
operating on the input feature space, followed by a patch attention module
$A^{p}(\cdot)$ to compute soft attention $\alpha$ over patches as follows:

$\tilde{g}_{i}=H(g_{i});\hskip
14.22636pt\alpha_{i}=A^{p}(\tilde{g}_{i});\hskip 14.22636pti\in\\{1,2,...N\\}$
(1)

where $A^{p}(\cdot)$ is a parameterized module with softmax activation. The
attention-scaled feature embeddings are input to the predictor $C(\cdot)$
which estimates the marginal contribution of each patch to the slide-level
task. Finally, these contributions are aggregated and activated with $\psi$ to
infer slide-level prediction $\hat{Y}_{g}$ as:

$\hat{Y}_{g}=\psi\Big{(}\sum_{i=1}^{N}C(\alpha_{i}\cdot\tilde{g}_{i})\Big{)}$
(2)

The MIL performs slide-level prediction while computing the contributions
through patch-level attentions. However, these attentions are too coarse for
pathological interpretability as they do not explain the underlying patterns
in pathologist-friendly terminologies.

###  3.2 Self-Interpretable MIL (SI-MIL)

As shown in Fig. 2a, along with the aforementioned conventional MIL as a
branch, SI-MIL consists of a Patch Attention-Guided Top-$K$ (PAG Top-$K$)
module and a SI branch. The PAG Top-$K$ module aims for a differentiable
selection of top $K$ patches identified in the MIL branch; thus enabling the
co-learning with the SI branch. This branch operates on these top $K$ patches,
by leveraging a feature attention module to linearly scale the corresponding
relevant pathological features. The patch-wise pathological features and
feature attention scores are subsequently aggregated by a linear predictor for
slide-level task. SI-MIL denotes the dual branch co-learning framework that
discriminates complex WSIs using a linear equation, advancing interpretability
by introducing feature-level insights while maintaining high performance and
complementing existing MILs. The details of the individual components are
described in the following sections.

PAG Top-$K$ patch selection module: This module leverages the patch attention
scores $\alpha$ from the conventional MIL branch to select the $K$ most
salient patches in a WSI. As the naive Top-$K$ operation is non-
differentiable, we use the differentiable perturbed Top-$K$ operation from
[11, 58]. This perturbed Top-$K$ operation is imperative to enable the co-
learning of both the branches: conventional MIL and SI branch. Following patch
selection, only the pathological features of the salient patches are utilized
in the subsequent steps. Therefore, the use of deep features in the MIL branch
does not hinder the interpretability of SI-MIL; rather it guides the selection
of informative patches, which is denoted as:

$S_{K}=\text{TopK}(\alpha,K)$ (3)

where $S_{K}$ denotes the indices of the selected top $K$ patches.

Feature Attention module: The $A^{f}(\cdot)$ module consists of a patch
feature mixing network and gated attention network. Their synergistic
integration forms a learnable feature selector without interfering with the
interpretability of SI-MIL. First, the pathological feature matrix
$M\in\mathbb{R}^{K\times d}$, corresponding to the $S_{K}$ patches, is
transposed and fed to a patch feature mixing network PF-Mixer, ${PF}(\cdot)$.
It contextualizes each value in $M^{T}$ with the top $K$ patches and $d$
features. In practice, ${PF}(\cdot)$ is implemented via MLP layers [59], with
separate layers dedicated to mixing spatial patch information and per-patch
feature information. Subsequently, gated attention network $G(\cdot)$
processes each row of the matrix $M^{T}\in\mathbb{R}^{d\times K}$
independently to determine the attention score $\beta_{j}$ for each feature
$d_{j}$, computed as:

$\beta_{j}=G({PF}(M^{T}));\hskip 14.22636ptj\in\\{1,2,...d\\}$ (4)

To enforce the model to be dependent on most salient features, we scale the
feature attention scores $\beta$ as follows: $\beta$ values are first scaled
using percentile $Pr_{\gamma}$ (where $\gamma$ is the $\gamma^{th}$
precentile) and standard deviation (std), and then sigmoid activated with a
hyper-parameter, temperature ($t$) as shown in Eqn. 5. This operation enforces
the $\beta$ values above $Pr_{\gamma}$ towards 1 and remaining towards 0,
thereby imposing sparsity in feature selection.

$\beta_{j}=\frac{\beta_{j}-Pr_{\gamma}(\beta)}{\text{std}(\beta)};\hskip
14.22636pt\beta_{j}=\frac{1}{1+e^{-\beta_{j}\times t}}$ (5)

These feature attention values are used to linearly scale the pathological
feature matrix $M$ such that the salient features are emphasized while
attenuating others:

$M^{{}^{\prime}}_{ij}=\beta_{j}\times M_{ij};\hskip
14.22636pti\in\\{1,2,...K\\};\hskip 2.84544ptj\in\\{1,2,...d\\}$ (6)

Note that even though $A^{f}(\cdot)$ includes non-linear operations to compute
$\beta$, the original feature space $M\in\mathbb{R}^{K\times d}$ is just
linearly scaled with $\beta$. $A^{f}(\cdot)$ paves the way for linear
prediction in the next stage, while preserving interpretability.

Linear Predictor and Aggregation: Following the attention scaling of the
pathological features corresponding to the $S_{K}$ patches, the features are
fed to a linear predictor $L(\cdot)$ characterized by weights $w(\cdot)$ and
bias $b$ as:

$M^{{}^{\prime\prime}}_{i}=\sum_{j=1}^{d}w_{j}M^{{}^{\prime}}_{ij}+b;\hskip
14.22636pti\in\\{1,2,...K\\}$ (7)

Finally, for slide-level prediction, the contributions
$M^{{}^{\prime\prime}}_{i}$ of the selected patches undergo an aggregation and
an activation $\psi$ as:

$\hat{Y}_{f}=\psi\Big{(}\sum_{i=1}^{K}M^{{}^{\prime\prime}}_{i}\Big{)}$ (8)

It can be observed that the WSI-level prediction in the SI branch can be
decomposed into a linear combination of feature attention scores $\beta$,
classifier weights $w(\cdot)$, and the pathological feature matrix
$M\in\mathbb{R}^{K\times d}$ of the top $K$ patches ($S_{K}$), given as:

$\hat{Y}_{f}=\psi\Big{(}\sum_{i=1}^{K}\sum_{j=1}^{d}w_{j}\beta_{j}M_{ij}+b\Big{)}$
(9)

Optimization: Given the true label $Y$ for a WSI, the predictions
$\hat{Y}_{g}$ from the MIL branch and $\hat{Y}_{f}$ from the SI branch, SI-MIL
is optimized using slide-level cross entropy losses $\mathcal{L}_{CE}$ for
both the predictions with respect to the true label. This joint optimization
tames the patch attention module to select the patches collaboratively in the
deep feature and pathological feature space. To enhance the performance of the
SI branch, a knowledge distillation loss $\mathcal{L}_{KD}$ is optimized based
on the mean squared error between $\hat{Y}_{f}$ and $\hat{Y}_{g}$.
$\mathcal{L}_{KD}$ enforces alignment in performance between the two branches.
The overall loss is computed as:

$\mathcal{L}=\mathcal{L}_{CE}(Y,\hat{Y}_{g})+\mathcal{L}_{CE}(Y,\hat{Y}_{f})+\lambda\mathcal{L}_{KD}(\hat{Y}_{g},\hat{Y}_{f})$
(10)

Note that during inference, prediction from any branch can be used. However to
enforce interpretability, the WSI-level prediction is obtained from the SI
branch, _i.e_., $\hat{Y}_{f}$ is considered for slide-level prediction and the
non-interpretable branch’s output $\hat{Y}_{g}$ is discarded.

###  3.3 WSI patch feature extraction

For each WSI, we extract patches ($p_{1}$, $p_{2}$, …$p_{N}$) and derive two
sets of features for each patch $p_{i}$, defined as:

  1. 1.

Deep features: We pretrain a ViT [13] through self-supervised learning on
patches from the WSIs, and use the ViT as feature extractor to encode a patch
$p_{i}$ into a deep feature vector $g_{i}\in\mathbb{R}^{D}$. Note that any
other pretrained or foundational model [62, 9, 61, 26, 63, 49] can be used for
patch encoding.

  2. 2.

Hand-crafted pathological features: We use HoVer-Net [17], pretrained on
PanNuke [16] dataset, to segment and classify nuclei into 5 classes in each
$p_{i}$. Then, pathologist-friendly features $f_{i}\in\mathbb{R}^{d}$ are
extracted to quantify nuclei morphology and spatial distribution properties in
$p_{i}$. These features can be grouped as:

Morphometric properties, _i.e_., intensity, shape, and texture, are computed
for all the nuclei in a patch, and are aggregated via statistical measures,
_i.e_., mean, standard deviation, skewness and kurtosis for each nuclei class.

Spatial distribution properties of different communities of nuclei types in a
patch are quantified using graph analysis and heterogeneity. The former uses
nuclei centroids to construct cell-graph and then extracts social network
analysis [66]-based features for each nucleus, followed by statistical
aggregation. These features capture properties such as degree of cohesiveness
and nuclei clustering. The latter quantifies the spatial interaction of
different nucleus class communities by using the nuclei centroids and class
labels. Entropy and infiltration based descriptors are leveraged for this
computation [42].

The comprehensive list detailing the different groups of features, along with
illustrative sample images are provided in supplementary.

##  4 Experiments: Prediction Performance

Here, we first describe the datasets and implementation details, common to
both performance and interpretability assessment. Then, we benchmark SI-MIL on
multiple WSI classification tasks. We conclude with ablation studies and
showcasing adaptability of SI-MIL to various MIL models.

###  4.1 Datasets and Implementation details

Datasets: We evaluate SI-MIL on three WSI datasets: TCGA-BRCA [33], TCGA-Lung
[3, 28], and TCGA-CRC [44]. TCGA-BRCA contains 910 diagnostic slides of two
breast cancer subtypes: invasive ductal carcinoma (IDC) and invasive lobular
carcinoma (ILC). TCGA-Lung contains 936 slides of two non-small cell lung
cancer subtypes: lung adenocarcinoma (LUAD) and lung squamous cell carcinoma
(LUSC). TCGA-CRC: includes 320 slides of colorectal cancer with low- or high-
mutation density for hypermutation. Additional details about train-test splits
are provided in the Supp. Sec. 7.

Patch and feature extraction: Patches of size $224\times 224$ at 5$\times$
magnification and corresponding $1792\times 1792$ at 40$\times$ are extracted
for each dataset. For deep features extraction, we pretrain ViT-S [13] with
DINO [8] on the 5$\times$ patches from the training splits of individual
datasets mentioned above. Pathological features are extracted on corresponding
patches at $40\times$.

MIL setting: Additive ABMIL [24] is adopted as the conventional MIL in this
study. $A^{p}(\cdot)$ and $A^{f}(\cdot)$ are gated attention modules adopted
from [36]. For all the MIL experiments, the batch size is set to 1 to handle
WSIs of variable bag sizes. For robustness, 5-fold cross-validation is
performed on the train split and the mean performance on the held-out test
split is reported. More implementation details are provided in the Supp. Sec.
8. Note that SI-MIL is evaluated with only DINO ViT-S features.
Experimentation with other deep features is left for future exploration.

###  4.2 Slide-level classification performance

In this section, we benchmark the WSI classification performance of SI-MIL in
terms of accuracy and area under the curve (AUC), which are the commonly
employed metrics to quantify the fidelity of interpretability algorithms [18].
Table 1 presents the classification performance of SI-MIL and the competing
baselines. In absence of WSI-level self-interpretable methods, we construct
interpretable baselines by perturbing SI-MIL under various settings. The
baselines can be grouped in terms of the types of employed features as
follows:

Baselines using deep features: These baselines denote training Additive ABMIL
with features from different pretrained deep feature extractors, _i.e_.,
ImageNet [12] supervised ViT-S (IN ViT-S), RetCCL [63], CTransPath [62], and
our pretrained DINO ViT-S. Although these baselines can render patch-level
contributions in terms of attention maps, one cannot entirely deduce the
reasoning behind these patch attentions, and cannot obtain feature-level
understanding due to their inherently non-interpretable characteristics.

Baselines using pathological features: These baselines denote training
Additive ABMIL with pathological features. To induce interpretability, we
train MIL with pathological features, referred as PathFeat. However, this
framework is non-interpretable as the projector $H(\cdot)$ maps the
pathological features into a non-interpretable deep feature space. Therefore,
we include a true interpretable baseline by training the MIL without
$H(\cdot)$.

2-stage training using pathological features: Here, we first train the
Additive ABMIL and extract top-$K$ attended patches for each WSI. Then, a
self-interpretable linear classifier using the pathological features from the
patches is trained. Specifically, we train the SI branch independent of the
MIL branch, _i.e_., without PAG Top-$K$. This is analogous to the post-hoc
analytical methods in [15, 10].

Table 1: Results indicate the mean of 5-fold cross-validation on test set. All
methods are trained with Additive ABMIL as base MIL. Int. denotes self-
interpretability of a method.

|  | Lung | BRCA | CRC
---|---|---|---|---
| Int. | Acc. | AUC | Acc. | AUC | Acc. | AUC
IN ViT-S | ✗ | 0.859 | 0.919 | 0.929 | 0.967 | 0.891 | 0.898
RetCCL | ✗ | 0.860 | 0.935 | 0.929 | 0.976 | 0.889 | 0.891
CTransPath | ✗ | 0.904 | 0.967 | 0.920 | 0.974 | 0.906 | 0.897
DINO ViT-S | ✗ | 0.896 | 0.957 | 0.937 | 0.974 | 0.904 | 0.897
PathFeat | ✗ | 0.830 | 0.888 | 0.885 | 0.950 | 0.886 | 0.818
PathFeat w/o $H(\cdot)$ | ✓ | 0.767 | 0.837 | 0.889 | 0.914 | 0.853 | 0.720
2-stage training | ✓ | 0.865 | 0.932 | 0.908 | 0.924 | 0.876 | 0.862
SI-MIL (ours) | ✓ | 0.884 | 0.941 | 0.944 | 0.968 | 0.884 | 0.910
Ablation study of SI-MIL components
w/o PAG Top-$K$ | ✓ | 0.859 | 0.936 | 0.915 | 0.922 | 0.876 | 0.869
w/o KD | ✓ | 0.853 | 0.915 | 0.932 | 0.951 | 0.878 | 0.830
w/o PAG Top-$K$ & KD | ✓ | 0.857 | 0.924 | 0.915 | 0.899 | 0.879 | 0.858

Results: As observed in Table 1, conventional MIL using pathological features,
and particularly the one without projector, performs considerably worse than
the methods using deep features. This accuracy-interpretability trade-off
often undermines the benefits of using interpretable frameworks/features. SI-
MIL aims to close this performance gap by utilizing deep feature-based
guidance. We find that SI-MIL, despite imposing a linear constraint (Eq. 9) on
predictions, elevates the performance of pathological features to be on par
with deep feature-based baselines.

Note that the results for RetCCL and CTransPath are potentially inflated as
the feature extractors were pretrained on the entire TCGA cohort, including
test splits used in our study. Thus, the DINO ViT-S and IN ViT-S baselines,
unaffected by the test split, provide a more accurate comparison.

Figure 3: Qualitative Patch-Feature importance report: In (a) and (b), we
present WSIs with overlaid attention heatmaps and the top two patches, along
with their nuclei maps. In (c), we demonstrate the mean contribution magnitude
of select representative features across the top $K$ patches employed in the
Self-Interpretable branch. Additionally, we display a feature density plot
that quantifies the distribution of features within the $K$ patches. For
brevity, we omit the y-axis. Given that these features are normalized, a curve
leaning towards the right indicates higher/positive values, while one towards
the left signifies lower/negative values, depending on the feature. Finally,
in (d), we illustrate, the description and visualization of representative
features in (c) with varying value.

Ablation studies: In Table 1, we mainly showcase the significance of the
meticulously designed components of SI-MIL. We show the implication of the PAG
Top-$K$ module by omitting the perturbed Top-$K$ selection and blocking
gradient flow from the SI branch to the MIL branch. In Table 1, we observe
that a non-differentiable approach degrades the performance. This indicates
that the most discriminative region identified by the MIL is potentially less
effective in the pathological feature space, thus highlighting the need to
find regions discriminative in both feature spaces for enhancing the
predictive power of SI branch. It can also be observed that in both settings,
with or without perturbed Top-$K$, knowledge distillation is instrumental in
enhancing the performance. $\mathcal{L}_{KD}$ acts as a regularizer for the SI
branch, pushing it to stay as close as to the high-performing MIL branch.
Additional ablations demonstrating the effect of varying $K$ in the PAG
Top-$K$ module, the number of PF-Mixer layers, and the percentile and
temperature for scaling $\beta$ are presented in the Supp. Sec. 9.

Adaptability of SI-MIL to other MILs: On the TCGA-BRCA dataset, we evaluate
the generalizability of SI-MIL by adapting to state-of-the-art MIL frameworks,
_i.e_., ABMIL [21], CLAM [36], and TransMIL [56], in the MIL branch. Results
in Table 2 establish that our SI-MIL extensions remain competitive with the
corresponding MIL methods using standalone DINO ViT-S features.

Table 2: Mean of 5-fold cross-validation for adapting SI-MIL with other MIL frameworks (additive versions [24]) on TCGA-BRCA. | DINO ViT-S | SI-MIL
---|---|---
MIL | Acc. | AUC | Acc. | AUC
ABMIL | 0.937 | 0.974 | 0.944 | 0.968
CLAM | 0.937 | 0.972 | 0.925 | 0.957
TransMIL | 0.934 | 0.936 | 0.929 | 0.933

##  5 Experiments and Results: Interpretability

In this section, we evaluate our SI-MIL model across various statistical
criteria, _i.e_., univariate and multivariate class-separability, and
desiderata of interpretability [18], _i.e_., user-friendliness and
faithfulness, focusing on both local slide-level and global cohort-level
interpretations. The user-friendliness metric evaluates how easily end-users,
_i.e_. pathologists, can understand and trust the model predictions, and the
faithfulness metric gauges the extent to which model’s explanations align with
the expert’s reasoning. The paper includes detailed analyses on TCGA-BRCA test
WSIs. Further analyses on TCGA-Lung and TCGA-CRC are presented in the Supp.
Sec. 12.

###  5.1 Local Interpretation: Slide-level

SI-MIL can explain model predictions at WSI-level without relying on post-hoc
methods [4]. Contrary to existing MIL [36, 56, 24], SI-MIL can produce both
patch- and feature-level explanations, due to the linear mapping between the
pathological features and output predictions. Fig. 3 presents aggregated
patch-feature importance reports generated by SI-MIL for two TCGA-BRCA WSIs,
elucidating the rationale behind the predictions. Below, we explain the setup
for generating such reports and then quantify their quality in terms of user-
friendliness and faithfulness.

WSI-level patch-feature importance report setup: Input WSIs with overlaid
patch-attention saliency maps, generated by the MIL branch are shown in Fig.
3a. Up next, Fig. 3b shows the informative top $K$ patches and their nuclei
predictions ($K=$2 for simplicity). The nuclear map identifies the nuclei
types and highlights their spatial organization in the tissue. Next, the
feature contributions across the top $K$ patches are detailed in Fig. 3c.
Recall that in Eqn. 9, $w_{j}\beta_{j}M_{ij}$ denotes the contribution of the
$i$-th patch and its $j$-th feature, where
$\sum_{i=1}^{K}w_{j}\beta_{j}M_{ij}$ infers the aggregated contribution of the
$j$-th feature towards WSI prediction. We present the mean contributions and
95% confidence intervals across $K$ patches, shown only for the three most
contributing features for simplicity. The negative and positive contributions
are indicative of class 0 (IDC) and class 1 (ILC), respectively, as activated
by sigmoid in Eq. 8. The feature distribution shows the range of the
corresponding normalized features across $K$ patches. The distribution
inclining towards left or right indicates low/negative values or high/positive
values of the feature, respectively. Looking at the distribution and
contribution together, if we have a positively inclining distribution and
negative contribution, then it means increasing the feature pushes the
prediction towards class 0. Fig. 3d illustrates a few features identified in
Fig. 3c, with the representative patches having low and high value of
corresponding features.

User-friendliness: We qualitatively evaluate the utility of the patch-feature
reports by an expert pathologist. First, we presented top $K$ ($=20$) patches
and corresponding nuclear maps for the WSIs in Fig. 3a to the expert. The
selected IDC and ILC patches demonstrated good agreement with class-specific
prior knowledge. The IDC patches contained coherent cancer cells forming
malignant glands, nests, or sheets with commentary about nuclear size, shape,
color, and chromatin texture; and the ILC patches showed infiltrating small
round cells in single file configurations. Afterwards, the top 10 contributing
pathological features and associated feature distributions, as identified by
SI-MIL in Fig. 3c, were evaluated by the expert to assess their correlation
with domain knowledge in classifying IDC and ILC. 90% and 80% of these
features for IDC and ILC WSIs in Fig. 3a were found relevant, respectively,
due to their strong association with cell cohesiveness, nuclear
hyperchromaticity, and morphology of cancerous nuclei properties. These
analyses helped the pathologist in reasoning with the model’s rationale and
developing trust in model’s predictions. Interestingly, the pathologist
commented on the utility of such feature-level relevance report in downstream
correlations with genomic and laboratory data.

Figure 4: Cohort-level Interpretation: Separability of top $K$ patches of WSIs
across classes in the pathological feature space. Multivariate and Univariate
analyses depict that the top $K$ patches selected by SI-MIL and their
pathological features are more separable.

Faithfulness: We evaluated the faithfulness of our reports by quantifying the
alignment of the top identified pathological features with pathologist’s
assessments. The evaluation involved the pathologist assigning relevance
scores to the top features. Specifically, we selected 10 WSIs each from IDC
and ILC, and generated patch-feature reports including top 10 contributing
pathological features. Then, the reports were analyzed and the features were
categorized into high-, moderate-, or non-relevant categories by the expert.
The mean and standard deviation of the number of features in each category are
reported separately in Table 3. Also, an aggregated percentage of the number
of features in each category is reported. The analysis shows that the majority
of the identified features are either highly or moderately relevant towards
correct classification and interpretability. Among the non-relevant features,
a few are interesting to be analyzed on larger cohorts to potentially discover
new diagnostic biomarkers. The selection of some of the non-relevant features
may also be due to certain misclassifications by HoVer-Net. This is left for
future exploration.

Table 3: Pathologist evaluation at slide-level for top contributing features’
relevancy for IDC and ILC classes in TCGA-BRCA. Agg. denotes aggregated
percentages of features belonging to three relevancy groups.

| Highly Relevant | Moderately Relevant | Non Relevant
---|---|---|---
IDC | 5.40 $\pm$ 1.43 | 2.10 $\pm$ 0.94 | 2.50 $\pm$ 1.28
ILC | 3.25 $\pm$ 0.97 | 3.75 $\pm$ 0.83 | 3.00 $\pm$ 1.12
Agg. | 44.5% | 28.3% | 27.2%

###  5.2 Global Interpretation: Cohort-level

In this section, we holistically analyze how SI-MIL interprets at a global
cohort-level and benefits over conventional MIL. Specifically, we perform
univariate and multivariate statistical analysis to measure class-separability
in the pathological feature space, inline with [15, 10, 43].

Univariate and Multivariate class-separability: Through global cohort-level
analysis, we demonstrate that SI-MIL, which includes the co-learning of MIL
and SI branches, optimizes the selection of more informative patches than
conventional MIL. During inference for both the models, we separately collect
the top $K$ attended patches across WSIs corresponding to the two classes in
TCGA-BRCA. Subsequently, we use the pre-extracted $d$ pathological features
for the selected patches, as described in Sec. 3.3. Formally, given $N_{1}$
and $N_{2}$ number of WSIs in the two classes, we construct pathological
feature matrices $F_{1}\in\mathbb{R}^{(N_{1}\times K)\times d}$ and
$F_{2}\in\mathbb{R}^{(N_{2}\times K)\times d}$ for both the models.

Multivariate analysis employs t-SNE [60] to project $F_{1}$ and $F_{2}$ into a
2D embedding space, as shown in Fig. 4a. Afterwards, we measure the class-
separability in terms of two metrics: (1) JSdiv@i, which entails fitting a 2D
Gaussian mixture model with $i$ components to each class and calculates the
Jansen-Shannon (JS) divergence between the two distributions; and (2)
Silhouette score [50], an unsupervised metric to evaluate the quality of
class-wise created clusters. Both the measures are distance-based metrics that
aim to highlight how separable the patches from the two classes are, in the
projected embedding space. To account for modeling variability, we report the
mean and standard deviation of the metrics across 5-fold cross-validation, as
presented in Sec. 4.1. It can be observed in the table in Fig. 4a that SI-MIL
consistently provides higher class-separability scores than conventional MIL
method. This can be attributed to the co-learning technique in SI-MIL, which
results in selecting more informative patches for individual classes that are
better separable in the pathological feature space.

Univariate analysis examines the class-separability of patches for individual
pathological features. For a given feature, _i.e_., a column in $F_{1}$ and
$F_{2}$, we create class-wise density distributions and measure the JS
divergence. For visual simplicity, we show the univariate analysis for the two
pathological features for both SI-MIL and the conventional MIL in Fig. 4b. We
can observe that the class-wise density distributions in SI-MIL are
significantly better separated than the MIL. This further supports our
argument of better patch selection in SI-MIL from multivariate analysis. For
an aggregated univariate analysis, we rank the features by the decreasing
order of JS divergence, and plot the median JS divergence against the
increasing number of features. Similar to multivariate analysis, we state the
mean and std of the medians across 5-fold cross-validation (Fig. 4b). We can
observe that SI-MIL provides significantly better median class-separability
for a good number of features, which strongly supports the enhanced quality of
selected patches while preserving pathological understanding.

###  5.3 Dataset contribution

We contribute a comprehensive dataset aimed at enhancing interpretability and
reproducibility in MIL research. It comprises of nuclei maps, pathological
features, and SI-MIL-generated patch-feature importance reports for 2.2K WSIs.
WSI processing and feature extraction involved significant computing resources
(details in Supp. Sec. 11). The complete list of pathological features,
including cell shape and texture properties, spatial configurations, and
interactions among different cell types, is detailed in the supplementary
material. We provide the key elements to enable researchers to further expand
on the already comprehensive set.

##  6 Conclusion

We present Self-Interpretable MIL, which not only augments model
interpretability by identifying salient regions and providing feature-level
contributions within these regions but also achieves high performance on
gigapixel WSI tasks. SI-MIL bridges the gap between AI-driven analysis and
pathologist-friendly reasoning, a first of its kind in histopathology. From an
evolutionary perspective, different cancers may share fundamentally similar
characteristics; the pathological features in SI-MIL can capture these
properties, possibly lending itself well to rare/unseen cancers. Future work
will also involve integration of LLM-driven pathological concepts in model
training.

## References

  * [1] The cancer genome atlas (tcga) research network.  https://www.cancer.gov/tcga.
  * Aflalo et al. [2022] Estelle Aflalo, Meng Du, Shao-Yen Tseng, Yongfei Liu, Chenfei Wu, Nan Duan, and Vasudev Lal.  Vl-interpret: An interactive visualization tool for interpreting vision-language transformers.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 21406–21415, 2022.
  * Albertina et al. [2016] B Albertina, M Watson, C Holback, R Jarosz, S Kirk, Y Lee, and J Lemmerman.  Radiology data from the cancer genome atlas lung adenocarcinoma [tcga-luad] collection.  _The Cancer Imaging Archive_ , 2016.
  * Arrieta et al. [2020] Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, et al.  Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai.  _Information fusion_ , 58:82–115, 2020.
  * Barbiero et al. [2023] Pietro Barbiero, Gabriele Ciravegna, Francesco Giannini, Mateo Espinosa Zarlenga, Lucie Charlotte Magister, Alberto Tonda, Pietro Lio, Frederic Precioso, Mateja Jamnik, and Giuseppe Marra.  Interpretable neural-symbolic concept reasoning.  _arXiv preprint arXiv:2304.14068_ , 2023.
  * Bilal et al. [2021] Mohsin Bilal, Shan E Ahmed Raza, Ayesha Azam, Simon Graham, Mohammad Ilyas, Ian A Cree, David Snead, Fayyaz Minhas, and Nasir M Rajpoot.  Development and validation of a weakly supervised deep learning framework to predict the status of molecular pathways and key mutations in colorectal cancer from routine histology images: a retrospective study.  _The Lancet Digital Health_ , 3(12):e763–e772, 2021.
  * Border and Sarder [2022] Samuel P Border and Pinaki Sarder.  From what to why, the growing need for a focus shift toward explainability of ai in digital pathology.  _Frontiers in Physiology_ , 12:821217, 2022.
  * Caron et al. [2021] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin.  Emerging properties in self-supervised vision transformers.  In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 9650–9660, 2021.
  * Chen et al. [2022a] Richard J Chen, Chengkuan Chen, Yicong Li, Tiffany Y Chen, Andrew D Trister, Rahul G Krishnan, and Faisal Mahmood.  Scaling vision transformers to gigapixel images via hierarchical self-supervised learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 16144–16155, 2022a.
  * Chen et al. [2022b] Richard J Chen, Ming Y Lu, Drew FK Williamson, Tiffany Y Chen, Jana Lipkova, Zahra Noor, Muhammad Shaban, Maha Shady, Mane Williams, Bumjin Joo, et al.  Pan-cancer integrative histology-genomic analysis via multimodal deep learning.  _Cancer Cell_ , 40(8):865–878, 2022b.
  * Cordonnier et al. [2021] Jean-Baptiste Cordonnier, Aravindh Mahendran, Alexey Dosovitskiy, Dirk Weissenborn, Jakob Uszkoreit, and Thomas Unterthiner.  Differentiable patch selection for image recognition.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 2351–2360, 2021.
  * Deng et al. [2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pages 248–255. Ieee, 2009.
  * Dosovitskiy et al. [2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  _arXiv preprint arXiv:2010.11929_ , 2020.
  * Espinosa Zarlenga et al. [2022] Mateo Espinosa Zarlenga, Pietro Barbiero, Gabriele Ciravegna, Giuseppe Marra, Francesco Giannini, Michelangelo Diligenti, Zohreh Shams, Frederic Precioso, Stefano Melacci, Adrian Weller, et al.  Concept embedding models: Beyond the accuracy-explainability trade-off.  _Advances in Neural Information Processing Systems_ , 35:21400–21413, 2022.
  * Fremond et al. [2023] Sarah Fremond, Sonali Andani, Jurriaan Barkey Wolf, Jouke Dijkstra, Sinéad Melsbach, Jan J Jobsen, Mariel Brinkhuis, Suzan Roothaan, Ina Jurgenliemk-Schulz, Ludy CHW Lutgens, et al.  Interpretable deep learning model to predict the molecular classification of endometrial cancer from haematoxylin and eosin-stained whole-slide images: a combined analysis of the portec randomised trials and clinical cohorts.  _The Lancet Digital Health_ , 5(2):e71–e82, 2023.
  * Gamper et al. [2019] Jevgenij Gamper, Navid Alemi Koohbanani, Ksenija Benet, Ali Khuram, and Nasir Rajpoot.  Pannuke: an open pan-cancer histology dataset for nuclei instance segmentation and classification.  In _Digital Pathology: 15th European Congress, ECDP 2019, Warwick, UK, April 10–13, 2019, Proceedings 15_ , pages 11–19. Springer, 2019.
  * Graham et al. [2019] Simon Graham, Quoc Dang Vu, Shan E Ahmed Raza, Ayesha Azam, Yee Wah Tsang, Jin Tae Kwak, and Nasir Rajpoot.  Hover-net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images.  _Medical image analysis_ , 58:101563, 2019.
  * Graziani et al. [2023] Mara Graziani, Lidia Dutkiewicz, Davide Calvaresi, José Pereira Amorim, Katerina Yordanova, Mor Vered, Rahul Nair, Pedro Henriques Abreu, Tobias Blanke, Valeria Pulignano, et al.  A global taxonomy of interpretable ai: unifying the terminology for the technical and social sciences.  _Artificial intelligence review_ , 56(4):3473–3504, 2023.
  * Hägele et al. [2020] Miriam Hägele, Philipp Seegerer, Sebastian Lapuschkin, Michael Bockmayr, Wojciech Samek, Frederick Klauschen, Klaus-Robert Müller, and Alexander Binder.  Resolving challenges in deep learning-based analyses of histopathological images using explanation methods.  _Scientific reports_ , 10(1):6423, 2020.
  * Huang et al. [2023] Zhi Huang, Federico Bianchi, Mert Yuksekgonul, Thomas J Montine, and James Zou.  A visual–language foundation model for pathology image analysis using medical twitter.  _Nature medicine_ , pages 1–10, 2023.
  * Ilse et al. [2018] Maximilian Ilse, Jakub Tomczak, and Max Welling.  Attention-based deep multiple instance learning.  In _International conference on machine learning_ , pages 2127–2136. PMLR, 2018.
  * Jaume et al. [2020] Guillaume Jaume, Pushpak Pati, Antonio Foncubierta-Rodriguez, Florinda Feroce, Giosue Scognamiglio, Anna Maria Anniciello, Jean-Philippe Thiran, Orcun Goksel, and Maria Gabrani.  Towards explainable graph representations in digital pathology.  _arXiv preprint arXiv:2007.00311_ , 2020.
  * Jaume et al. [2021] Guillaume Jaume, Pushpak Pati, Behzad Bozorgtabar, Antonio Foncubierta, Anna Maria Anniciello, Florinda Feroce, Tilman Rau, Jean-Philippe Thiran, Maria Gabrani, and Orcun Goksel.  Quantifying explainers of graph neural networks in computational pathology.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 8106–8116, 2021.
  * Javed et al. [2022] Syed Ashar Javed, Dinkar Juyal, Harshith Padigela, Amaro Taylor-Weiner, Limin Yu, and Aaditya Prakash.  Additive mil: intrinsically interpretable multiple instance learning for pathology.  _Advances in Neural Information Processing Systems_ , 35:20689–20702, 2022.
  * [25] Jayashree Kalpathy-Cramer, Andrew Beers, Artem Mamonov, Erik Ziegler, Rob Lewis, Andre Botelho Almeida, Gordon Harris, Steve Pieper, David Clunie, Ashish Sharma, et al.  Crowds cure cancer: Data collected at the rsna 2017 annual meeting.  _The Cancer Imaging Archive. DOI_ , 10:K9.
  * Kapse et al. [2023] Saarthak Kapse, Srijan Das, Jingwei Zhang, Rajarsi R Gupta, Joel Saltz, Dimitris Samaras, and Prateek Prasanna.  Attention de-sparsification matters: Inducing diversity in digital pathology representation learning.  _arXiv preprint arXiv:2309.06439_ , 2023.
  * Kawaguchi [2016] Kenji Kawaguchi.  Deep learning without poor local minima.  _Advances in neural information processing systems_ , 29, 2016.
  * Kirk et al. [2016] S Kirk, Y Lee, P Kumar, J Filippini, B Albertina, M Watson, and J Lemmerman.  Radiology data from the cancer genome atlas lung squamous cell carcinoma [tcga-lusc] collection.  _The Cancer Imaging Archive_ , 2016.
  * Koh et al. [2020] Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang.  Concept bottleneck models.  In _International conference on machine learning_ , pages 5338–5348. PMLR, 2020.
  * Laurent and Brecht [2018] Thomas Laurent and James Brecht.  Deep linear networks with arbitrary loss: All local minima are global.  In _International conference on machine learning_ , pages 2902–2907. PMLR, 2018.
  * Li et al. [2021] Bin Li, Yin Li, and Kevin W Eliceiri.  Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 14318–14328, 2021.
  * Li et al. [2023] Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao.  Llava-med: Training a large language-and-vision assistant for biomedicine in one day.  _arXiv preprint arXiv:2306.00890_ , 2023.
  * Lingle et al. [2016] W Lingle, BJ Erickson, ML Zuley, R Jarosz, E Bonaccio, J Filippini, and N Gruszauskas.  Radiology data from the cancer genome atlas breast invasive carcinoma [tcga-brca] collection.  _The Cancer Imaging Archive_ , 10:K9, 2016.
  * Liu et al. [2018] Yang Liu, Nilay S Sethi, Toshinori Hinoue, Barbara G Schneider, Andrew D Cherniack, Francisco Sanchez-Vega, Jose A Seoane, Farshad Farshidfar, Reanne Bowlby, Mirazul Islam, et al.  Comparative molecular analysis of gastrointestinal adenocarcinomas.  _Cancer cell_ , 33(4):721–735, 2018.
  * Lu et al. [2021a] Cheng Lu, Can Koyuncu, German Corredor, Prateek Prasanna, Patrick Leo, XiangXue Wang, Andrew Janowczyk, Kaustav Bera, James Lewis Jr, Vamsidhar Velcheti, et al.  Feature-driven local cell graph (flock): new computational pathology-based descriptors for prognosis of lung cancer and hpv status of oropharyngeal cancers.  _Medical image analysis_ , 68:101903, 2021a.
  * Lu et al. [2021b] Ming Y Lu, Drew FK Williamson, Tiffany Y Chen, Richard J Chen, Matteo Barbieri, and Faisal Mahmood.  Data-efficient and weakly supervised computational pathology on whole-slide images.  _Nature biomedical engineering_ , 5(6):555–570, 2021b.
  * Lu et al. [2023a] Ming Y Lu, Bowen Chen, Drew FK Williamson, Richard J Chen, Ivy Liang, Tong Ding, Guillaume Jaume, Igor Odintsov, Andrew Zhang, Long Phi Le, et al.  Towards a visual-language foundation model for computational pathology.  _arXiv preprint arXiv:2307.12914_ , 2023a.
  * Lu et al. [2023b] Ming Y Lu, Bowen Chen, Andrew Zhang, Drew FK Williamson, Richard J Chen, Tong Ding, Long Phi Le, Yung-Sung Chuang, and Faisal Mahmood.  Visual language pretrained multiple instance zero-shot transfer for histopathology images.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 19764–19775, 2023b.
  * Madabhushi and Lee [2016] Anant Madabhushi and George Lee.  Image analysis and machine learning in digital pathology: Challenges and opportunities.  _Medical image analysis_ , 33:170–175, 2016.
  * Mahinpei et al. [2021] Anita Mahinpei, Justin Clark, Isaac Lage, Finale Doshi-Velez, and Weiwei Pan.  Promises and pitfalls of black-box concept learning models.  _arXiv preprint arXiv:2106.13314_ , 2021.
  * Margeloiu et al. [2021] Andrei Margeloiu, Matthew Ashman, Umang Bhatt, Yanzhi Chen, Mateja Jamnik, and Adrian Weller.  Do concept bottleneck models learn as intended?  _arXiv preprint arXiv:2105.04289_ , 2021.
  * Martinelli and Rapsomaniki [2022] Adriano Luca Martinelli and Maria Anna Rapsomaniki.  Athena: analysis of tumor heterogeneity from spatial omics measurements.  _Bioinformatics_ , 38(11):3151–3153, 2022.
  * McKenzie et al. [2022] Andrew T McKenzie, Gabriel A Marx, Daniel Koenigsberg, Mary Sawyer, Megan A Iida, Jamie M Walker, Timothy E Richardson, Gabriele Campanella, Johannes Attems, Ann C McKee, et al.  Interpretable deep learning of myelin histopathology in age-related cognitive impairment.  _Acta Neuropathologica Communications_ , 10(1):131, 2022.
  * Network et al. [2012] Cancer Genome Atlas Network et al.  Comprehensive molecular characterization of human colon and rectal cancer.  _Nature_ , 487(7407):330, 2012.
  * Nguyen and Hein [2017] Quynh Nguyen and Matthias Hein.  The loss surface of deep and wide neural networks.  In _International conference on machine learning_ , pages 2603–2612. PMLR, 2017.
  * Oikarinen et al. [2023] Tuomas Oikarinen, Subhro Das, Lam M Nguyen, and Tsui-Wei Weng.  Label-free concept bottleneck models.  _arXiv preprint arXiv:2304.06129_ , 2023.
  * Pati et al. [2023] Pushpak Pati, Guillaume Jaume, Zeineb Ayadi, Kevin Thandiackal, Behzad Bozorgtabar, Maria Gabrani, and Orcun Goksel.  Weakly supervised joint whole-slide segmentation and classification in prostate cancer.  _Medical Image Analysis_ , 89:102915, 2023.
  * Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In _International conference on machine learning_ , pages 8748–8763. PMLR, 2021.
  * Riasatian [2020] Abtin Riasatian.  Kimianet: Training a deep network for histopathology using high-cellularity.  Master’s thesis, University of Waterloo, 2020.
  * Rousseeuw [1987] Peter J Rousseeuw.  Silhouettes: a graphical aid to the interpretation and validation of cluster analysis.  _Journal of computational and applied mathematics_ , 20:53–65, 1987.
  * Rudin [2019] Cynthia Rudin.  Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.  _Nature machine intelligence_ , 1(5):206–215, 2019.
  * Rymarczyk et al. [2022] Dawid Rymarczyk, Adam Pardyl, Jarosław Kraus, Aneta Kaczyńska, Marek Skomorowski, and Bartosz Zieliński.  Protomil: Multiple instance learning with prototypical parts for whole-slide image classification.  In _Joint European Conference on Machine Learning and Knowledge Discovery in Databases_ , pages 421–436, 2022.
  * Sadafi et al. [2023] Ario Sadafi, Oleksandra Adonkina, Ashkan Khakzar, Peter Lienemann, Rudolf Matthias Hehr, Daniel Rueckert, Nassir Navab, and Carsten Marr.  Pixel-level explanation of multiple instance learning models in biomedical single cell images.  In _International Conference on Information Processing in Medical Imaging_ , pages 170–182. Springer, 2023.
  * Salahuddin et al. [2022] Zohaib Salahuddin, Henry C Woodruff, Avishek Chatterjee, and Philippe Lambin.  Transparency of deep neural networks for medical image analysis: A review of interpretability methods.  _Computers in biology and medicine_ , 140:105111, 2022.
  * Saltz et al. [2018] J Saltz, R Gupta, L Hou, T Kurc, P Singh, V Nguyen, D Samaras, KR Shroyer, T Zhao, R Batiste, et al.  Tumor-infiltrating lymphocytes maps from tcga h&e whole slide pathology images [data set].  _Cancer Imaging Arch_ , 2018.
  * Shao et al. [2021] Zhuchen Shao, Hao Bian, Yang Chen, Yifeng Wang, Jian Zhang, Xiangyang Ji, et al.  Transmil: Transformer based correlated multiple instance learning for whole slide image classification.  _Advances in neural information processing systems_ , 34:2136–2147, 2021.
  * Sureka et al. [2020] Mookund Sureka, Abhijeet Patil, Deepak Anand, and Amit Sethi.  Visualization for histopathology images using graph convolutional neural networks.  In _2020 IEEE 20th international conference on bioinformatics and bioengineering (BIBE)_ , pages 331–335. IEEE, 2020.
  * Thandiackal et al. [2022] Kevin Thandiackal, Boqi Chen, Pushpak Pati, Guillaume Jaume, Drew FK Williamson, Maria Gabrani, and Orcun Goksel.  Differentiable zooming for multiple instance learning on whole-slide images.  In _European Conference on Computer Vision_ , pages 699–715. Springer, 2022.
  * Tolstikhin et al. [2021] Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al.  Mlp-mixer: An all-mlp architecture for vision.  _Advances in neural information processing systems_ , 34:24261–24272, 2021.
  * Van der Maaten and Hinton [2008] Laurens Van der Maaten and Geoffrey Hinton.  Visualizing data using t-sne.  _Journal of machine learning research_ , 9(11), 2008.
  * Vorontsov et al. [2023] Eugene Vorontsov, Alican Bozkurt, Adam Casson, George Shaikovski, Michal Zelechowski, Siqi Liu, Philippe Mathieu, Alexander van Eck, Donghun Lee, Julian Viret, et al.  Virchow: A million-slide digital pathology foundation model.  _arXiv preprint arXiv:2309.07778_ , 2023.
  * Wang et al. [2022] Xiyue Wang, Sen Yang, Jun Zhang, Minghui Wang, Jing Zhang, Wei Yang, Junzhou Huang, and Xiao Han.  Transformer-based unsupervised contrastive learning for histopathological image classification.  _Medical image analysis_ , 81:102559, 2022.
  * Wang et al. [2023] Xiyue Wang, Yuexi Du, Sen Yang, Jun Zhang, Minghui Wang, Jing Zhang, Wei Yang, Junzhou Huang, and Xiao Han.  Retccl: clustering-guided contrastive learning for whole-slide image retrieval.  _Medical image analysis_ , 83:102645, 2023.
  * Yang et al. [2023] Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison-Burch, and Mark Yatskar.  Language in a bottle: Language model guided concept bottlenecks for interpretable image classification.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 19187–19197, 2023.
  * Yuksekgonul et al. [2022] Mert Yuksekgonul, Maggie Wang, and James Zou.  Post-hoc concept bottleneck models.  _arXiv preprint arXiv:2205.15480_ , 2022.
  * Zamanitajeddin et al. [2021] Neda Zamanitajeddin, Mostafa Jahanifar, and Nasir Rajpoot.  Cells are actors: Social network analysis with classical ml for sota histology image classification.  In _Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part VIII 24_ , pages 288–298. Springer, 2021.
  * Zhang et al. [2021] Jingwei Zhang, Ke Ma, John Van Arnam, Rajarsi Gupta, Joel Saltz, Maria Vakalopoulou, and Dimitris Samaras.  A joint spatial and magnification based attention framework for large scale histopathology classification.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 3776–3784, 2021.
  * Zhang et al. [2023] Jingwei Zhang, Saarthak Kapse, Ke Ma, Prateek Prasanna, Joel Saltz, Maria Vakalopoulou, and Dimitris Samaras.  Prompt-mil: Boosting multi-instance learning schemes via task-specific prompt tuning.  _arXiv preprint arXiv:2303.12214_ , 2023.
  * Zhang et al. [2017] Zizhao Zhang, Yuanpu Xie, Fuyong Xing, Mason McGough, and Lin Yang.  Mdnet: A semantically and visually interpretable medical image diagnosis network.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 6428–6436, 2017.
  * Zheng et al. [2022] Yi Zheng, Rushin H Gindra, Emily J Green, Eric J Burks, Margrit Betke, Jennifer E Beane, and Vijaya B Kolachalama.  A graph-transformer for whole slide image classification.  _IEEE transactions on medical imaging_ , 41(11):3003–3015, 2022.

Supplementary Material

In this supplementary material, details are provided on the following:

  * •

Dataset details (additional) (7)

  * •

Implementation details (additional) (8)

  * •

SI-MIL ablation studies: hyperparameter sensitivity (9)

  * •

SI-MIL ablation studies: model components (10)

  * •

Dataset contribution details (additional) (11)

  * •

Local interpretability analysis (additional) (12)

  * •

Global interpretability analysis (additional) (13)

  * •

Top-$K$ comparative analysis (14)

  * •

Hand-crafted pathological feature extraction (15)

##  7 Dataset details (additional)

We benchmark our SI-MIL on three WSI datasets, namely TCGA-BRCA, TCGA-Lung,
and TCGA-CRC. SI-MIL necessitates pathological features for interpretable
prediction. We use HoVer-Net for segmenting and classifying the nuclei, and
afterwards computed hand-crafted pathological features. Since HoVer-Net is
trained exclusively on 40$\times$ magnification patches, our analysis is
confined to WSIs having 40$\times$ magnification. This ensures accurate nuclei
prediction and thereby meaningful pathological feature extraction.

TCGA-BRCA is split into 825 training (653 IDC, 172 ILC) and 85 testing (67
IDC, 18 ILC) WSIs following [9]. TCGA-Lung dataset is split into 744 training
(388 LUAD, 356 LUSC) and 192 testing (96 LUAD, 96 LUSC) WSIs following DSMIL
[31]. For TCGA-CRC, following [34, 6], we use the first three folds for
training, _i.e_., 241 WSIs (38 hypermutated, 203 not) and the fourth for
testing, _i.e_., 79 slides (12 hypermutated, 67 not) with 40$\times$
filtering. The patch extraction process implemented in our study follows the
methodology outlined in the aforementioned DSMIL repository [31].

##  8 Implementation details (additional)

###  8.1 Deep feature extractors

We compared SI-MIL against different baselines that include training Additive
ABMIL using different types of patch features. Details of the patch feature
extractors are presented as follows:

IN ViT-S: We train the Additive ABMIL using features extracted by a popular
ImageNet-supervised model, specifically ViT-Small [13] model pre-trained using
ImageNet dataset [12]. The model extracts a feature embedding of size $D=384$
for each WSI patch.

RetCCL: We adopted a state-of-the-art feature extractor [63] pre-trained using
pathology images. This benchmarks our trained feature extractor, described in
Sec. 4.2. This model extracts a feature embedding of size $D=2048$.

CTransPath: Similar to RetCCL, we benchmarked against a Transformer-based
feature extractor pre-trained using pathology images [62]. The resulting patch
embeddings are of size $D=768$.

It is important to note that CTransPath and RetCCL were pre-trained on the
pan-TCGA [1] dataset, and our evaluated datasets are subset of this dataset.
Therefore, these models were pre-trained using the WSIs in our test dataset,
which can potentially result in inflated performances during classification.
Though benchmarked, these models may not be suitable for reliable comparisons
in our study.

DINO ViT-S: For a reliable comparison, we used DINO [8] to pre-train ViT-Small
models for each dataset (TCGA-BRCA, TCGA-Lung, and TCGA-CRC), using the
dataset-specific training splits as provided in the Supp Sec. 7 For pre-
training, we used the default hyperparameter values of DINO [8], while using
only two global crops. These pre-trained models extract a feature embedding of
size $D=384$ for each WSI patch. One RTX 8000 GPU is utilized for pretraining
the ViT-S with a batch size of 256.

###  8.2 SI-MIL

Hyperparameter tuning is performed with a range of learning rates
$\in\\{1e^{-3},2e^{-3},1e^{-4},2e^{-4}\\}$ and weight decays
$\in\\{1e^{-2},5e^{-3}\\}$. By default, $\\#$PF-Mixer layers $=$4,
$\lambda=20$, $K=20$, $\gamma=0.75$, and $t=3$. Additionally, $d=246$ except
for in TCGA-Lung where $d=203$ as annotations for only 4 (instead of 5) cell
types are available for HoVer-Net classification in the Lung dataset. For both
the predictors $L(\cdot)$ and $C(\cdot)$, we use the sigmoid activation
($\psi$), since our tasks involve binary classification. Note that
$\mathcal{L}_{KD}$ is utilized with stop-gradient since the goal is to align
the performance of the Self-Interpretable branch to be close to high
performing conventional MIL branch in SI-MIL. All MIL experiments are
performed on one RTX 8000 GPU.

###  8.3 Interpretability analysis setup

For interpretability analysis, we compare the separability of the top $K$
patches in the pathological feature space between the conventional MIL and SI-
MIL (refer to Figure 4). To ensure a fair comparison, we select WSIs from the
held-out test set where both MIL methods result in correct predictions.

Note that, we employ 5-fold cross-validation on the training split and held
out the test set. We chose the best-performing fold for both local
(visualization and pathologist relevancy score experiment) and global
(visualization) interpretability analysis for the MIL methods. However, for
multivariate class-separability scores (refer to Figure 4), we report the
median and standard deviation from all 5-folds. Similarly, we report the
median and standard deviation of Jensen-Shannon (JS) divergence across all
5-folds in Figure 4.

###  8.4 SI-MIL complexity analysis:

The mitigation of the trade-off between performance and interpretability by
SI-MIL can be attributed to the choice of pathological features and the SI-MIL
design choices, instead of merely an increase in the number of model
parameters. It can be justified by comparing the size and performance of SI-
MIL with the competing baselines. The number of model parameters in SI-MIL is
625K, while those in conventional MIL with DINO/IN ViT-S, CTransPath and
ReTCCL are 345K, 985K, and 5.25M, respectively. Despite the differences in
model sizes, SI-MIL results in comparable performance with respect to the
competing baselines, as shown in Table 1.

##  9 SI-MIL ablation studies: hyperparameters

In this section, we provide studies of SI-MIL hyperparameters on TCGA-BRCA
dataset. Particularly, these ablations demonstrate the effect of varying $K$
in the PAG Top-$K$ module, the number of PF-Mixer layers, and the percentile
and temperature for scaling $\beta$.

Effect of varying $K$ in the PAG Top-$K$ module: In Figure 5, we illustrate
the impact of varying $K$ on SI-MIL performance. We observe that a larger
value of $K$ leads to a significant drop in performance compared to the
default $K=20$. This decrease may be attributed to an increase in irrelevant
noisy patches, which makes it difficult for the model to classify WSIs in the
pathological feature space.

Figure 5: PAG Top-$K$ module ablation

Effect of varying number of PF-Mixer layers: SI-MIL’s performance is generally
robust across various values of the number of PF-Mixer layers, but experiences
a performance drop for very high values, _e.g_., #PF-Mixer layers $=6$ (Figure
6). This decline can be attributed to potential overfitting induced as a
result of higher number of layers.

Figure 6: PF-Mixer module layers ablation

Effect of percentile and temperature for scaling $\beta$: In Figure 7, we show
the variation in performance of SI-MIL with respect to the percentile value
($Pr_{\gamma}$) and temperature ($t$) for scaling the feature attention values
$\beta$ in eq. 5. “None” in Figure 7 refers to the absence of percentile and
standard deviation scaling.

$Pr_{\gamma}$ controls the percentage of features ($d$) that have a positive
value before being fed to the sigmoid activation in eq. 5. The temperature
parameter ($t$) determines the sharpness of this curve, with a high value
indicating that most values deviate from zero before being fed to the sigmoid.
Thus, having high $Pr_{\gamma}$ and high $t$ leads to a very sparse selection
of features. Since our goal is to interpret the prediction of WSI, it is
beneficial to explain the prediction in terms of the contribution of “few”
most discriminative features. Note that the absence of $Pr_{\gamma}$ scaling
and/or low temperature allows the model to use a large number of features for
its prediction; thus making it harder to interpret the predictions. Therefore,
the main goal is to have higher values of $Pr_{\gamma}$ and $t$, while
maintaining a good SI-MIL performance.

In Figure 7, we can observe that having no $Pr_{\gamma}$ scaling generally
results in the best performance, whereas a very high value, such as
$Pr_{\gamma}=0.9$, performs poorly in most cases. We find that having a
slightly lower $Pr_{\gamma}=0.75$ and $t=3$ establishes an optimal balance, by
enforcing adequate sparsity while still performing efficiently.

Figure 7: $\beta$ scaling ablation

##  10 SI-MIL ablation studies: components

In Table 1, we demonstrate the variations in performance when ablating
different components of SI-MIL. Similarly, in Table 4, we present additional
experiments on the impact of different SI-MIL components.

We observed that omitting the Feature Attention module $A^{f}(\cdot)$ results
in better performance compared to using it without the PF-Mixer network
${Mix}(\cdot)$, though both scenarios underperform relative to the proposed
SI-MIL. This indicates that $A^{f}(\cdot)$, which softly selects features,
requires contextualization among the patches and features before highlighting
or attenuating specific features within this module. Without appropriate
contextualization, processing each feature row in matrix $M^{T}$ independently
leads to suboptimal decisions by $A^{f}(\cdot)$ and reduces performance.

We further investigate the necessity of deep features in SI-MIL. For this
purpose, we substituted deep features with pathological features in the
conventional MIL branch, thereby using the same pathological features in both
SI-MIL branches. As shown in Table 4, the performance declines with or without
$\mathcal{L}_{KD}$ when replacing deep features, underscoring the importance
of employing potent deep features to guide the Self-Interpretable branch in
SI-MIL.

Table 4: Results indicate the mean of 5-fold cross-validation on test set. All
methods are trained with Additive ABMIL as the base MIL. Int. denotes self-
interpretability of a method.

|  | Lung | BRCA | CRC
---|---|---|---|---
| Int. | Acc. | AUC | Acc. | AUC | Acc. | AUC
PathFeat | ✗ | 0.830 | 0.888 | 0.885 | 0.950 | 0.886 | 0.818
PathFeat w/o $H(\cdot)$ | ✓ | 0.767 | 0.837 | 0.889 | 0.914 | 0.853 | 0.720
2-stage training | ✓ | 0.865 | 0.932 | 0.908 | 0.924 | 0.876 | 0.862
SI-MIL (ours) | ✓ | 0.884 | 0.941 | 0.944 | 0.968 | 0.884 | 0.910
Ablation study of SI-MIL components
w/o $A^{f}(\cdot)$ | ✓ | 0.853 | 0.935 | 0.939 | 0.981 | 0.871 | 0.857
w/o ${Mix}(\cdot)$ | ✓ | 0.838 | 0.915 | 0.925 | 0.953 | 0.866 | 0.863
w/ PathFeat only | ✓ | 0.863 | 0.936 | 0.911 | 0.942 | 0.876 | 0.836
w/ PathFeat only & w/o KD | ✓ | 0.847 | 0.911 | 0.911 | 0.945 | 0.853 | 0.781

##  11 Dataset contribution details (additional)

We contribute a unique comprehensive dataset aimed at enhancing
interpretability and reproducibility in MIL research. The dataset encompasses
nuclei maps, pathological features, and SI-MIL-generated patch-feature
importance reports for over 2,200 WSIs. It covers multiple organs and cancer
types, including Lung (lung adenocarcinoma vs. lung squamous cell carcinoma),
Breast (invasive ductal vs. invasive lobular carcinoma), and Colon (low vs.
high mutation). This diverse collection facilitates in-depth studies across
various cancer types, providing a valuable resource for advancements in
interpretable MIL methodologies.

Successful translation of AI tools to the clinic hinges upon the
interpretability and trustworthiness of the tools. This dataset will serve as
a critical asset for both the medical vision and digital pathology
communities, facilitating the exploration of new research directions in the
development of interpretable AI techniques for computational pathology. A
significant obstacle in digital pathology research has been the intensive
resource requirements for extracting features that possess clear geometric and
physical significance, and which are interpretable by pathologists. The
dataset creation involved analyzing gigapixel WSIs at 40$\times$
magnification, leveraging HoVer-Net [17] for cell segmentation and
classification, followed by extracting pathological features and feature
importance scores detailed in Sec. 3.3 and Sec. 5, respectively. Processing
each WSI required $\sim$2 hours, divided between GPU-based cell map prediction
and CPU-based pathological feature extraction. Employing three RTX 8000 GPUs
and a 40-core CPU with 500GB RAM, the total processing amounted to $\sim$4400
hours ($\approx$60 days). We provide the comprehensive set to enable further
research.

In view of $\sim$2 TB memory foorprint of HoVer-Net nuclei maps and the
processed pathological features, we intend to host this dataset on TCIA
Analysis Results, akin to other popular preprocessed datasets [25, 55].

The dataset will be released under the Creative Commons Attribution-
NonCommercial 4.0 International License (CC BY-NC 4.0). It permits the
sharing, copying, and redistribution in any medium or format, as well as
adaptation, remixing, transforming, and building upon the material for non-
commercial purposes. Appropriate credit must be given, a link to the license
must be provided, and any changes made should be indicated.

##  12 Local interpretability analysis (additional)

Here, we present additional predictions (refer to Sec. 5) for WSIs from other
datasets, _i.e_., TCGA-Lung and TCGA-CRC. Please note that the predictions for
all WSIs in the evaluated datasets will be released as part of the contributed
dataset. Qualitative patch-feature importance reports for TCGA-Lung and TCGA-
CRC are illustrated in the upper and lower half of Figure 8, respectively.

Figure 8: Qualitative Patch-Feature importance report: In (a) and (b), we
present WSIs with overlaid attention heatmaps and the top two patches, along
with their nuclei maps. In (c), we demonstrate the mean contribution magnitude
of select representative features across the top $K$ patches employed in the
Self-Interpretable branch. Additionally, we display a feature density plot
that quantifies the distribution of features within the $K$ patches. For
brevity, we omit the y-axis. Given that these features are normalized, a curve
leaning towards the right indicates higher/positive values, while one towards
the left signifies lower/negative values, depending on the feature. Finally,
in (d), we illustrate, the description of representative features in (c).

##  13 Global interpretability analysis (addition)

Here, we present global interpretability analysis (refer to Sec. 5) for
patches from the test set WSIs. We include only the WSIs that were correctly
predicted by both the conventional MIL and SI-MIL, to ensures a fair
comparison, as described in Sec. 8. Cohort-level interpretation for TCGA-Lung
and TCGA-CRC are illustrated in the upper and lower half of Figure 9,
respectively.

Figure 9: Cohort-level Interpretation: Separability of top $K$ patches of WSIs
across classes in the pathological feature space. Multivariate and Univariate
analyses depict that the top $K$ patches selected by SI-MIL and their
pathological features are more separable.

##  14 Top-$K$ comparative analysis

In this section, we demonstrate how the Self-Interpretable branch of SI-MIL
tames the patch attention map of conventional deep MIL. Specifically, Figures
10 and 11 compare the spatial attention maps generated after the training of
conventional MIL (_i.e_., without pathological features) and SI-MIL, which
integrates both the conventional MIL and Self-Interpretable branches. We
proceed to visualize the top $K=20$ patches from both MIL methods,
categorizing them into groups based on whether they are common or distinct
between the methods.

In Figure 10, we contrast the top $K=20$ patch selection of our Self-
Interpretable MIL (SI-MIL) with conventional MIL in analyzing invasive ductal
carcinoma (IDC) WSIs. SI-MIL and conventional MIL share 6 out of 20 patches,
but differ in the remaining 14. While conventional MIL often chooses patches
near the dermis, featuring IDC with smooth connective areas and occasionally
normal glands, SI-MIL targets patches indicative of malignancy, marked by
malignant cancerous ducts with large, distorted nuclei. This difference,
especially evident in the unique patches of SI-MIL, underscores its focus on
diagnostically relevant areas like malignant glands with compressed lumens and
hyperchromatic nuclei, contrasted against the tissue highlighted by
conventional MIL. SI-MIL’s emphasis is on patches comprising 70-80% of
malignant features, including dense pink-colored cancer-associated stroma,
aligning with its goal of accurate diagnosis.

In the context of invasive lobular carcinoma (ILC), early detection is crucial
due to its rapid spread and poor long-term survival outcomes, necessitating
clear differentiation from invasive ductal carcinoma (IDC). In Figure 11, we
observe an absence of common patches between the top $K=20$ attended patches
of both MIL methods. Our method, in contrast to conventional MIL,
distinctively focuses on invasive single file chains, often found at the
periphery of the tumor bulk or the invasive front, which are more
characteristic of ILC. This is in contrast to the conventional MIL’s emphasis
on patches with high cellularity within the tumor bulk. The rapid spread of
lobular cancer is evident in its infiltration through various tissues, and
unlike IDC, which often presents as glandular structures with clear
separations between tumorous and connective nuclei, ILC is characterized by
discohesive arrangements, leading to single file patterns with a notable
mixing of tumor nuclei with connective nuclei.

Figure 10: TCGA-BRCA Invasive Ductal Carcinoma (IDC) sample. Refinement of the
patch attention map by the Self-Interpretable branch, transitioning from
conventional MIL to SI-MIL. Figure 11: TCGA-BRCA Invasive Lobular Carcinoma
(ILC) sample. Refinement of the patch attention map by the Self-Interpretable
branch, transitioning from conventional MIL to SI-MIL.

##  15 Hand-crafted pathological feature extraction

In Sec. 3.3, we briefly discussed the categories of handcrafted pathological
features, such as Morphometric and Spatial distribution properties. This
section provides a detailed description of these features, accompanied by
visualizations to elucidate the significance of their geometrical and physical
meaning in computational pathology. We further refine these features into
three categories, based on the studies from which they were adopted:
Morphometric properties (from FLocK [35]), Graph-based Social Network Analysis
[66], and Spatial heterogeneity properties [42].

###  15.1 Feature categories

We employ HoVer-Net [17] to segment and classify nuclei in each WSI patch
$p_{i}$, using the model trained on PanNuke. The classified nuclei include
Neoplastic epithelial, Connective, Inflammatory, Necrosis, and Non-neoplastic
epithelial classes. Subsequently, image-processing tools are used to quantify
the properties and spatial distribution of nuclei in each patch. Next we
provide a detailed description of each feature within these categories.

####  15.1.1 Morphometric properties

In a patch $p_{i}$, we extract 10 morphometric properties for each segmented
nucleus as outlined in Table 5. To represent the entire patch, these nuclei-
level features are aggregated using 4 statistical properties: mean, standard
deviation, skewness, and kurtosis. This aggregation is performed separately
for each of the 5 nuclei classes identified by HoVer-Net. Additionally, the
number of nuclei in each class is included as a feature.

Consequently, this results in a total of 205 patch-level aggregated
morphometric properties: $10\times 4\times 5$ for the morphometric properties
aggregated across the four statistics and five nuclei classes, plus five for
the count of nuclei in each class.

Group | Feature
---|---
Shape | Area
Eccentricity
Roundness
Orientation
Morphology | Mean of Intensity
Standard Deviation of Intensity
Contrast of Texture
Dissimilarity of Texture
Homogeneity of Texture
Energy of Texture
Table 5: Description of extracted morphometric properties for each segmented
nuclei

In computational pathology, these 205 morphometric properties from each patch
$p_{i}$ provide a holistic tissue profile. These features encapsulate key
morphological characteristics of nuclei, crucial for pathological assessment.
By employing statistics like mean, standard deviation, skewness, and kurtosis,
we gain insights into the variability, asymmetry, and tailedness of the
nuclei’s morphological properties within each patch. Separately analyzing
these features for each of the five nuclei classes as identified by HoVer-Net
enriches the model’s understanding of the tissue heterogeneity and cellular
composition. Additionally, counting nuclei per class quantifies cellular
composition, further enriching the diagnostic value in computational
pathology.

####  15.1.2 Graph-based Social Network Analysis

In a patch $p_{i}$, we construct a graph based on the centroids of nuclei and
quantify the properties of this network. Drawing inspiration from [66], we
initially create a k-nearest neighbor graph (with $k=6$) using the centroid
locations of each segmented nucleus, irrespective of their classes.
Subsequently, we extract 4 traditional social network analysis properties for
each nucleus, as detailed in Table 6. This is followed by statistical
aggregation to patch-level using mean, standard deviation, skew, kurtosis, and
max. This results in total 20 aggregated Social Network features.

Feature
---
Degree
Degree centrality
Clustering coefficient
Closeness centrality
Table 6: Description of extracted social network analysis properties for each
nuclei

The Degree and Degree Centrality metric in our study provides insight into the
number of direct connections a nucleus has within the tissue network,
illuminating its level of interaction. This is pivotal in understanding nuclei
communication and behavior in various pathological states. The Clustering
Coefficient is another key measure, offering insights into the extent of
interconnectivity among a nucleus’s neighbors. This can reveal localized
nuclei clusters, a feature often observed in certain pathological conditions.
Lastly, Closeness Centrality assesses the average shortest distance from a
nucleus to all others, aiding in identifying nuclei that are central or
isolated within the tissue architecture. This comprehensive analysis of nuclei
organization and interaction patterns through these SNA features is crucial
for an in-depth understanding of the tissue’s pathology.

####  15.1.3 Spatial Heterogeneity properties

This feature group goes beyond analyzing just the centroids of nuclei; it also
incorporates their classes to assess the spatial heterogeneity of various
nuclei communities within a patch [42]. Heterogeneity is quantified at both
global and local levels in each patch.

Global level: A range of entropy-based descriptors and k-function metrics are
utilized, examining all segmented nuclei to evaluate the uniformity versus
randomness in their spatial distribution. These global heterogeneity
descriptors are listed in Table 7.

Local level: Following the methodology in [42], we construct a k-nearest
neighbor graph (with $k=6$) using the nuclei. For each nucleus, entropy and
interaction-based properties are extracted, focusing on immediate neighbors. A
local interaction-score is then aggregated at the patch level, as per the
process in [42]. Additionally, skewness of entropy property distribution
across nuclei is computed. This skewness metric discerns whether most nuclei
have lower, medium, or higher entropy values, thus offering a detailed view of
cellular interactions and complexity. This local-level approach highlights the
intermixing of different nuclei communities, taking into account their spatial
relationships, an aspect overlooked by global entropy-based descriptors. These
local-level features are enumerated in Table 8.

This results in total 21 Spatial Heterogeneity features (9 Global and 12
Local). For an in-depth explanation and visualization of these features, we
direct readers to the seminal work by [42], which extensively explores these
methodologies and their implications.

Group | Feature
---|---
Global Entropy | Global Shannon index
Global Simpson index
Global max entropy
Global Richness (number of cell-types present)
Graph modularity with cell types as community
k-function | Neoplastic cells: k-function at radius 224 pixels
Neoplastic cells: k-function at radius 448 pixels
Neoplastic cells: k-function at radius 672 pixels
Neoplastic cells: k-function at radius 896 pixels

Table 7: Global Spatial Heterogeneity Descriptors

Group | Feature
---|---
Local Entropy [42] | Skewness of cells’ local Shannon index
Skewness of cells’ local Simpson index
Skewness of cells’ local max-entropy
Skewness of cells’ local richness
Local Interaction score [42] | Mixing of neoplastic cells in inflammatory cells’ region
Mixing of neoplastic cells in connective cells’ region
Mixing of neoplastic cells in necrosis cells’ region
Mixing of neoplastic cells in non-neoplastic epithelial cells’ region
Mixing of inflammatory cells in neoplastic cells’ region
Mixing of connective cells in neoplastic cells’ region
Mixing of necrosis cells in neoplastic cells’ region
Mixing of non-neoplastic epithelial cells in neoplastic cells’ region

Table 8: Local Spatial Heterogeneity Descriptors

For instance, clustered arrangements of different nuclei communities typically
result in lower local-level entropy, as the neighboring nuclei are mainly from
the same class. Conversely, intermixed arrangements lead to higher local-level
entropy due to the diversity of neighboring nuclei classes. However, at the
global level, these differing arrangements may yield similar entropy values if
the overall count of each nuclei class remains constant, despite their
distinct spatial distributions. This highlights the importance of analyzing
spatial heterogeneity at both local and global levels to capture the full
complexity of cellular arrangements in tissue pathology.

###  15.2 Normalization

In this study, we implemented a two-step normalization process for all
handcrafted pathological features. The first step addresses potential
inaccuracies in nuclei segmentation and classification by HoVer-Net, using a
binning operation. Each feature within a patch is assessed based on its
percentile relative to other patches in the training split of a WSI task.
These percentiles are then categorized into 10 discrete bins, ranging from the
0-10th to the 90-100th percentile, effectively shifting the scale of features
from absolute values to a relative range from very-low to very-high. This
approach transforms feature values into a robust and interpretable format
across different patches. The second step involves mean and standard deviation
normalization, once again using the training split data. This step centers the
data around zero, optimizing it for effective processing by neural networks.

We emphasize that the normalization process alters the scale of the features.
For instance, the skewness properties listed in Table 8 would typically be
near 0, negative, or positive in their absolute scale. However, after mean-
standard deviation and binning normalization, the scale of skewness may shift,
with a 0 skew potentially appearing on either the positive or negative side.
Hence, for a more accurate interpretation of our predictions in the local
slide-level interpretable predictions (refer to Figure 3), it is crucial to
consider this scaling effect. Readers should interpret the features as being
generally in the lower or higher range and then conceptually approximate these
back to their absolute scale. This approach ensures a more nuanced
understanding of the predictions post-normalization.

###  15.3 Feature Visualization

In the following figures (Figures 12 to 16), we present a visual exploration
of some representative features by showcasing patches with low and high values
of these features. Each figure is accompanied by a detailed caption that
elucidates the feature in the context of the patches, providing insights into
what constitutes low and high values with respect to that specific feature.
This visual representation aids in understanding the impact of these features
on the tissue’s pathology and offers a deeper perspective on how they manifest
in different patches.

Note that the terms ‘cell’ and ‘nucleus’ are used interchangeably. However,
since the imaging modality is H&E, all the features actually pertain to
nuclei.

Figure 12: Neoplastic Cells: Mean of Eccentricity. As illustrated, in the
patches under the column named ‘Low’, there are round cancer cells (the larger
ones), whereas on the right side, under the column named ‘High’, elliptical
cells are present, indicating a higher mean of eccentricity. In
histopathology, this feature refers to the average deviation of cancer cells
from a perfect circular shape. A higher mean eccentricity, as observed in the
‘High’ column, suggests more elliptical cells, often associated with more
aggressive or advanced cancer forms.  Figure 13: Neoplastic Cells: Mean of
Intensity Standard Deviation. As illustrated, in the patches under the column
named ‘Low’, there are cancer cells (the larger ones) with uniform intensity,
thus the standard deviation is low for each cell, leading to a low mean
intensity standard deviation. Whereas on the right side, under the column
named ‘High’, the cells exhibit anisochromasia, indicating a higher mean
intensity standard deviation. In histopathology, this feature refers to the
average deviation of cancer cells from homogeneous intensity. A higher value
of this feature, as observed in the ‘High’ column, suggests more
anisochromasia, often associated with more aggressive or advanced cancer
forms.  Figure 14: Standard Deviation of Cells’ Degree. As illustrated, in the
patches under the column named ‘Low’, there is a homogeneous distribution of
cells of all types. For this feature, we first construct a k-nearest neighbor
graph from cells’ centroid and then calculate the cell’s degree for each cell.
Therefore, a homogeneous distribution leads to each cell having a similar
degree, resulting in a lower value of standard deviation. Whereas on the right
side, under the column named ‘High’, the cells are much more randomly
distributed (disorganized), with grouping in some areas and sparse cells in
others. This leads to some cells having a higher degree and others lower,
resulting in a high standard deviation of cells’ degree in a patch. In
histopathology, this feature loosely refers to cohesive versus non-cohesive or
homogeneous versus heterogeneous distribution in a spatial context. Figure 15:
Graph Modularity with Cell Types as Community. As illustrated, in the patches
under the column named ‘Low’, cancer cells (in red) co-occur in close spatial
proximity with other cell types such as connective (in green) and inflammatory
cells (in blue). This results in interconnections among different cell classes
when constructing a graph for this feature, leading to low graph modularity.
Whereas on the right side, under the column named ‘High’, cells of different
classes/communities are more distinctly separated and grouped, resulting in
more connections within the same community in the k-nearest neighbor graph,
leading to higher graph modularity. In histopathology, this feature can serve
as a proxy for distinguishing ductal versus single file line patterns in IDC
versus ILC classification in TCGA-BRCA. Figure 16: Infiltration of Connective
Cells in Neoplastic Cells’ Region. In contrast to the scenario presented in
Figure 15, the ‘High’ column patches display cancer cells (colored in red)
closely intermingled with connective cells (colored in green). This proximity
results in more interactions between these cell types in the graph-based
analysis of this feature, leading to a marked increase in the infiltration of
connective cells within the neoplastic area. On the other hand, in the ‘Low’
column, there is a clearer segregation and clustering of the two cell classes,
manifesting in reduced connectivity between them in the k-nearest neighbor
graph, and consequently, lower levels of infiltration. In histopathological
analysis, this characteristic can be instrumental in differentiating ductal
cancers, which show minimal infiltration by other cell types, from invasive
patterns characterized by a significant presence of connective cells within
the neoplastic areas. This explanation is also applicable to features like the
Infiltration of Inflammatory Cells in Neoplastic Cells’ Region.