Lucas Dedieu1<EMAIL_ADDRESS>
Nicolas Nerrienet1<EMAIL_ADDRESS>
Adrien Nivaggioli1<EMAIL_ADDRESS>
Clara Simmat1<EMAIL_ADDRESS>
Marceau Clavel1<EMAIL_ADDRESS>
Arnaud Gauthier1,2<EMAIL_ADDRESS>
Stéphane Sockeel1<EMAIL_ADDRESS>
Rémy Peyret1<EMAIL_ADDRESS>
1 Primaa, Paris, France
2 Diagnostic and Theranostic Medicine Division, Department of Pathology,
Institut Curie, PSL University, Paris, France

# Contrastive-Based Deep Embeddings for Label Noise-Resilient Histopathology
Image Classification

###### Abstract

Recent advancements in deep learning have proven highly effective in medical
image classification, notably within histopathology. However, noisy labels
represent a critical challenge in histopathology image classification, where
accurate annotations are vital for training robust deep learning models.
Indeed, deep neural networks can easily overfit label noise, leading to severe
degradations in model performance. While numerous public pathology foundation
models have emerged recently, none have evaluated their resilience to label
noise. Through thorough empirical analyses across multiple datasets, we
exhibit the label noise resilience property of embeddings extracted from
foundation models trained in a self-supervised contrastive manner. We
demonstrate that training with such embeddings substantially enhances label
noise robustness when compared to non-contrastive-based ones as well as
commonly used noise-resilient methods. Our results unequivocally underline the
superiority of contrastive learning in effectively mitigating the label noise
challenge. Code is publicly available at
https://github.com/LucasDedieu/NoiseResilientHistopathology

###### keywords:

Histopathology, Image Classification, Label Noise, Contrastive Learning, Deep
Embeddings, Foundation Models

##  1 Introduction

Histopathology, the microscopic examination of tissue samples, plays a crucial
role in the diagnosis, prognosis, and treatment of various diseases, including
cancer. With the rise of digital pathology thanks to whole-slide image (WSI)
scanners, image analysis and deep learning are becoming part of
histopathologists’ routine. As the workload of histopathologists has grown
significantly in previous decades [Garcia et al. (2020)], deep learning-based
image analysis now plays a major role in increasing clinical workflow
efficiency [Dawson (2022)]. To be effective, training such deep neural
networks (DNNs) requires large image datasets with reliable labels. However,
in the context of medical imaging and histopathology in particular, clean data
are rare and expensive, requiring expert labeling campaigns. Thus, one of the
key challenges in histopathology image analysis is the presence of label
noise, which refers to errors or inaccuracies in the annotations provided for
the training data. These inaccuracies, stemming from inter-observer
variability, imperfect segmentation of tissue regions, inherent ambiguity in
the biological features, and omission errors, impede the development of
reliable deep learning models. Indeed, it has been proven that DNNs can easily
overfit noisy labels [Li et al. (2018), Zhang et al. (2016)], leading to
severe degradations in model performance and thus potentially misleading
clinical decisions.

To mitigate this challenge, various approaches have been proposed in the
literature, ranging from robust loss functions to novel training methodologies
and label cleaning strategies. In more recent works, self-supervised-based
methods also emerged. Nevertheless, the quest for noise-tolerant models in the
presence of label noise remains an ongoing pursuit.

In this paper, we aim to comprehensively evaluate the robustness of deep
embeddings extracted from pretrained backbones specifically designed for
histopathology image classification. We compare the noise tolerance of models
trained solely on these features to those trained on the original images and
assess their performance on multiple histopathology datasets. Our key
contributions are: (1) We demonstrate that classifiers trained on contrastive
deep embeddings exhibit improved robustness to label noise compared to those
trained on the original images using state-of-the-art methods. This highlights
the inherent noise resilience of these features. (2) We compare the noise
robustness of contrastive embeddings to alternative self-supervised embeddings
extracted from non-contrastive backbones, demonstrating the superior ability
of contrastive-based backbones to handle noisy labels.

##  2 Related Work

###  2.1 Learning with Noisy Labels Methods

In image classification, a multitude of approaches have been explored to
enhance the robustness of learning algorithms in the presence of noisy labels.

#### Label Cleaning Strategies:

CleanNet [Lee et al. (2017)] uses a small, clean dataset to generate feature
vectors for each class, comparing them with query image vectors to determine
label accuracy. Rank Pruning [Northcutt et al. (2017)] focuses on high-
confidence data points for classifier training, potentially overlooking
challenging samples common in medical tasks. To overcome this, Zhu et al.
(2022) propose a two-phase hard sample aware noise robust learning algorithm
consisting in an easy/hard/noisy detection scheme combined with a noise
supressing and hard enhancing scheme.

#### Robust Loss Functions:

Other methods focus on changing the loss function, such as Ghosh et al. (2017)
that showed that the categorical cross entropy (CCE) loss is sensitive to
label noise while the mean absolute error (MAE) loss is robust. However,
training a network under MAE loss leads to underfitting and thus Zhang and
Sabuncu (2018) proposed generalized cross entropy (GCE), a noise-robust loss
function that can be seen as a generalization of MAE and CCE. In the same way,
Ma et al. (2020) showed that every loss can be noise-robust through simple
normalization, but may lead to an underfitting problem. To tackle this
challenge, the authors introduce a synergistic boosting framework that enables
the development of a novel family of loss functions, each possessing the
theoretical guarantee of robustness while maintaining efficient learning.

#### Training Procedures:

Numerous studies also explored novel training approaches to address the issue
of label noise. Co-teaching Han et al. (2018) proposed a new deep learning
paradigm that involves maintaining two neural networks in parallel and
selecting instances with low loss values for cross-training. Improving this
work, Co-teaching+ Yu et al. (2019) bridges ”update by disagreement” strategy
with the original Co-teaching. Also using two networks, DivideMix Li et al.
(2020) employs a mixture model to dynamically partition the training data into
clean and noisy subsets, enabling semi-supervised learning on both. To address
confirmation bias, the two networks are trained separately, each using the
other network’s dataset division.

###  2.2 Self-Supervised-Based Methods

Some recent works leverage advances in self-supervised learning (SSL) to deal
with noisy labels. SSL is a paradigm in which models are trained from
unlabeled data using pretext tasks. By solving these pretext tasks, models
learn meaningful representations of the data and the acquired knowledge is
then transferred to the primary task of classification or segmentation.
Tackling label noise, Co-learning Tan et al. (2021) trains two classifier
heads with a shared feature encoder and regularizes the model with both the
intrinsic similarity and the structural similarity. Kurian et al. (2022) use a
contrastive learning framework and feature aggregating memory banks to
identify and increase the emphasis on clean training samples. Jiang et al.
(2023) integrate contrastive learning and intra-group attention mixup
techniques into standard supervised learning. Khanal et al. (2023) showed that
model pretraining with SimCLR Chen et al. (2020a) consistently yields the best
results in presence of noisy labels on two medical datasets.

The reasons behind the enhancement of label noise resilience through
contrastive learning are still not fully understood, and as far as our
knowledge extends, only one study Xue et al. (2022) has delved into the
theoretical understanding of this achievement. The authors demonstrated that
contrastive learning yields a representation matrix characterized by a
significant gap between the prominent singular values and the remaining ones,
along with a considerable alignment between the prominent singular vectors and
the accurate labels. Aforementioned characteristics enable a linear classifier
head trained on these representations to adeptly learn clean labels while
minimizing overfitting to the noisy ones.

###  2.3 Pathology Foundation Models

The emergence of SSL has been particularly impactful in the domain of
histology, where obtaining large amounts of accurately labeled data is
challenging and expensive. In the last few years, many foundation models
trained in a self-supervised way have been developed. Wang et al. (2022, 2021)
introduced CTransPath, a feature extractor based on the swin transformer Liu
et al. (2021), trained on TCGA Weinstein et al. (2013) and PAIP Kim et al.
(2021) using semantically-relevant contrastive learning (SRCL), a novel SSL
technique adapted from MoCo v3 Chen et al. (2021) specifically designed for
pathology applications. They also presented RetCCL Wang et al. (2023), a
ResNet-50 He et al. (2015) model trained on the same two datasets using their
clustering-guided contrastive learning (CCL) SSL technique, which was based on
MoCo He et al. (2020). Filiot et al. (2023) evaluated various vision
transformer (ViT) variants Kolesnikov et al. (2021) on TCGA using the iBOT
framework Zhou et al. (2022) and mask image modeling, a new paradigm of SSL
coming from the world of natural language processing (NLP) applied to vision.
Authors named their most effective ViT-B variant ”Phikon”. Lunit Kang et al.
(2023) conducted a benchmark of several SSL techniques, including Barlow Twins
Zbontar et al. (2021), SwAV Caron et al. (2020), MoCo v2 Chen et al. (2020b),
and DINO Caron et al. (2021) (self-distillation), for pathology by training
them on TCGA. Hua et al. (2023) trained Pathoduet on TCGA using newly-
introduced pretext tokens and later task raisers to explicitly utilize certain
relations between images and showed better results than CTransPath on some
patch classification tasks. In recent months, several other pathology
foundation models have emerged, trained on much larger datasets Chen et al.
(2023); Vorontsov et al. (2023); Dippel and Feulner (2024); Campanella and
Kwan (2023); Lu et al. (2023). Regrettably, we could not incorporate these
models into our research due to their proprietary nature.

##  3 Proposed Approach

###  3.1 Problem Formulation

Given a $K$-class image classification problem, let
$\mathcal{X}\subset\mathbb{R}^{d}$ be the image space, and let
$\mathcal{Y}=\\{1,...,K\\}$ be the class labels. In a typical learning scheme,
we are given training data,
$S=\\{(\mathbf{x},y)^{(i)}\\}_{i=1}^{N}\in(\mathcal{X}\times\mathcal{Y})^{N}$.
Classification aims to learn a function $f:\mathcal{X}\rightarrow\mathcal{Y}$
(represented by a DNN) that maps the input space to the label space. Training
model $f$ involves finding optimal parameters $\theta$ minimizing the
empirical risk, i.e.,
$\theta:=\arg\min_{\theta}\sum_{i=1}^{N}\mathcal{L}(f(\mathbf{x_{i}}),y_{i}))$,
where $\mathcal{L}(f(\mathbf{x}),y)$ is the loss of $f$ with respect to $y$.

In presence of uniform label noise, the noisy training data available to the
classifier is $S_{\eta}=\\{(\mathbf{x},\hat{y})^{(i)}\\}_{i=1}^{N}$, where
$\eta\in[0,1]$ denotes the uniform noise rate in the dataset, and

$\hat{y}_{i}=\left\\{\begin{array}[]{rcl}y_{i}&\mbox{with
probability}&(1-\eta)\\\ k,k\neq y_{i}&\mbox{with
probability}&p_{\eta}=\frac{\eta}{K-1}\\\ \end{array}\right.$ (1)

######  Remark 3.1.

We do not consider a uniform noise rate $\eta>\frac{K-1}{K}$ as, for balanced
dataset, it would mean that for each class $k$, $\exists c\neq k,$ where $c$
has more data labeled as class $k$ than $k$.

In asymmetric label noise scenarios, the flipping probabilities of each class
are conditioned by a transition matrix $T$, thus $p_{\eta}=T_{y_{i},k}$
represents the probability of transitioning from true label $y_{i}$ to noisy
label $k$. Asymmetric label noise tends to provide a more realistic
representation of real-world noise scenarios as certain classes may be more
prone to confusion due to visual similarity or contextual ambiguity.

###  3.2 Learning with Contrastive-Based Deep Embeddings

To mitigate label noise, we propose an approach based on contrastive-based
deep embeddings. We employ a contrastive-pretrained backbone to extract
embeddings from images and then exploit these embeddings and corresponding
noisy labels to train a linear classifier head. Here, $f=h\circ g$ with
$g:\mathcal{X}\rightarrow\mathcal{B}$, the frozen backbone parameterized by
$\theta_{g}$ and $h:\mathcal{B}\rightarrow\mathcal{Y}$, the classifier head
parameterized by $\theta_{h}$. $\mathcal{B}$ represents the latent space of
embeddings. The main advantage of this training approach is that it allows to
use the same pre-trained feature extractor $g$ for multiple datasets, and only
train a separate classifier $h_{i}$ for each dataset $i=1,...,M$. It enables
significantly faster trainings since the latent space $\mathcal{B}$ and
parameters $\theta_{h}$ are much smaller than the image space $\mathcal{X}$
and $\theta$.

##  4 Experiments

In this section, we propose a comparison of different methods for handling
noisy labels. We benchmark four image-based methods: a baseline, GCE, Active-
Passive loss (APL), and DivideMix. In addition, we compare eight deep
embedding methods based on eight foundation models. These include four
contrastive histology backbones: RetCCL, CTransPath, PathoDuet, and Lunit
Barlow Twins (BT); two non-contrastive histology backbones: Phikon and Lunit
DINO; and two ViT-B models trained on ImageNet Deng et al. (2009) with MoCo v3
(contrastive) and iBOT (non-contrastive). We include the two ImageNet models
to also investigate the performance of contrastive and non-contrastive
embeddings derived from backbones in unrelated domains. The experimental
settings are detailed in appendix A.

###  4.1 Datasets and Pre-processing

We validate the effectiveness of our proposed approach by assessing its
performance on six public benchmark datasets, namely NCT-CRC-HE-100K Kather et
al. (2018), PatchCamelyon Veeling et al. (2018), BACH Polónia et al. (2020),
MHIST Wei et al. (2021), LC25000 Borkowski et al. (2019) and GasHisSDB Hu et
al. (2022). tab:dataset, summarizes all dataset details. We add more
information about these datasets and the pre-processing used in appendix B.

Table 1: Dataset details. 222: set obtained from train set split, 333: set labellized by a pathologist expert for this study. Dataset | Patch Size | Train | Validation | Test | $K$
---|---|---|---|---|---
NCT-CRC-HE-100k | 224x224 | 80,000 |  20,000222 | 7,180 | 9
PatchCamelyon | 96x96 | 262,144 | 32,768 | 32,768 | 2
BACH | 2048x1536 | 320 |  80222 |  83333 | 4
MHIST | 224x224 | 1740 |  435222 | 977 | 2
LC25000 | 768x768 | 16,000 |  4000222 |  5000222 | 5
GasHisSDB | 160x160 | 21,303 |  5325222 |  6656222 | 2

###  4.2 Results

The results under uniform label noise, as depicted in tab:sym, highlight the
efficacy of training with deep embeddings. Across nearly all the datasets and
noise rates scenarios, these methods consistently match or surpass
performances of image-based approaches. Particularly noteworthy is the
observation that, for noise rates $\eta>0$, classifiers trained with
contrastive embeddings exhibit superior performance compared to their non-
contrastive counterparts. This phenomenon is graphically illustrated in
fig:curves, providing a finer-grained insight into the impact of noise rates.
Across all datasets, the accuracies of Phikon and Lunit-DINO (non-contrastive)
consistently exhibit sharper declines compared to others.

Table 2: Average test accuracies with standard deviation under different
uniform label noise ratios (%, 2 runs for image methods, 4 runs for deep
embedding methods). 111:Foundation model trained with contrastive learning.

Method | NCT-CRC-HE-100k | PatchCamelyon
---|---|---
|  $\eta=0$ |  $\eta=0.2$ |  $\eta=0.4$ |  $\eta=0.6$ |  $\eta=0.8$ |  $\eta=0$ |  $\eta=0.2$ |  $\eta=0.4$
Baseline |  96.7$\pm$0.1 |  90.4$\pm$0.7 |  83.2$\pm$1.7 |  70.9$\pm$6.1 |  35.5$\pm$0.3 |  87.3$\pm$2.1 |  85.9$\pm$1.0 |  73.1$\pm$1.8
GCE |  96.8$\pm$0.2 |  96.1$\pm$0.5 |  94.8$\pm$0.4 |  93.4$\pm$0.2 |  82.9$\pm$5.3 |  85.5$\pm$1.1 |  85.2$\pm$1.6 |  82.8$\pm$1.0
APL |  96.4$\pm$0.1 |  96.6$\pm$0.1 |  96.2$\pm$0.1 |  95.1$\pm$0.2 |  8.8$\pm$0.1 |  89.1$\pm$0.1 |  86.4$\pm$0.4 |  81.2$\pm$3.3
DivideMix |  97.1$\pm$0.1 |  96.9$\pm$0.1 |  96.4$\pm$0.2 |  96.3$\pm$0.2 |  94.7$\pm$1.7 |  90.3$\pm$0.1 |  87.4$\pm$0.2 |  87.0$\pm$0.7
Phikon |  94.3$\pm$0.6 |  92.2$\pm$1.4 |  88.4$\pm$5.2 |  82.6$\pm$4.0 |  40.9$\pm$11.2 |  86.4$\pm$0.7 |  82.5$\pm$1.3 |  77.6$\pm$2.2
RetCCL111 |  94.5$\pm$0.1 |  95.1$\pm$0.2 |  95.2$\pm$0.1 |  94.8$\pm$0.2 |  93.6$\pm$0.3 |  85.8$\pm$0.7 |  84.5$\pm$0.2 |  82.7$\pm$0.6
Lunit-DINO |  95.4$\pm$0.5 |  95.0$\pm$0.6 |  93.3$\pm$3.1 |  86.6$\pm$8.3 |  61.6$\pm$8.8 |  88.1$\pm$1.4 |  85.5$\pm$0.3 |  80.1$\pm$2.6
Lunit-BT111 |  94.3$\pm$0.3 |  94.7$\pm$0.3 |  94.4$\pm$0.3 |  94.1$\pm$0.4 |  92.3$\pm$0.8 |  91.0$\pm$0.2 |  90.7$\pm$0.2 |  89.2$\pm$0.1
CTransPath111 |  97.4$\pm$0.2 |  97.4$\pm$0.2 |  97.4$\pm$0.2 |  96.9$\pm$0.1 |  95.1$\pm$0.3 |  89.0$\pm$0.5 |  87.0$\pm$0.8 |  85.2$\pm$1.5
PathoDuet111 |  96.1$\pm$0.4 |  96.3$\pm$0.2 |  95.9$\pm$0.3 |  95.5$\pm$0.2 |  93.3$\pm$0.8 |  87.1$\pm$0.5 |  87.0$\pm$0.2 |  85.8$\pm$0.4
iBOT |  93.3$\pm$0.5 |  90.1$\pm$1.2 |  84.6$\pm$4.4 |  78.3$\pm$6.2 |  45.1$\pm$10.2 |  83.2$\pm$0.5 |  77.8$\pm$1.9 |  68.9$\pm$1.9
MoCo111 |  93.2$\pm$0.5 |  93.0$\pm$1.1 |  90.8$\pm$1.4 |  88.8$\pm$0.4 |  86.0$\pm$0.4 |  81.2$\pm$0.3 |  80.1$\pm$0.4 |  78.6$\pm$0.5

Method | BACH | MHIST
---|---|---
|  $\eta=0$ |  $\eta=0.2$ |  $\eta=0.4$ |  $\eta=0.6$ |  $\eta=0$ |  $\eta=0.2$ |  $\eta=0.4$
Baseline |  78.2$\pm$2.1 |  65.1$\pm$3.4 |  41.3$\pm$6.9 |  40.5$\pm$6.3 |  82.9$\pm$0.9 |  72.1$\pm$3.1 |  60.2$\pm$0.7
GCE |  76.9$\pm$0.2 |  68.5$\pm$1.8 |  61.3$\pm$1.5 |  52.3$\pm$2.7 |  84.3$\pm$0.6 |  76.9$\pm$0.6 |  59.2$\pm$6.7
APL |  77.5$\pm$0.6 |  73.1$\pm$0.6 |  64.6$\pm$2.3 |  51.7$\pm$4.9 |  83.8$\pm$0.9 |  72.1$\pm$8.9 |  68.2$\pm$0.6
DivideMix |  79.6$\pm$0.1 |  73.8$\pm$2.0 |  64.9$\pm$3.1 |  50.9$\pm$2.1 |  84.2$\pm$0.2 |  80.9$\pm$0.4 |  72.2$\pm$1.4
Phikon |  77.4$\pm$1.3 |  69.3$\pm$2.2 |  55.1$\pm$9.7 |  40.0$\pm$4.8 |  84.5$\pm$0.5 |  75.0$\pm$0.9 |  57.9$\pm$4.8
RetCCL111 |  73.5$\pm$0.5 |  71.7$\pm$0.3 |  68.7$\pm$1.7 |  61.0$\pm$1.3 |  81.2$\pm$0.7 |  78.1$\pm$1.2 |  73.0$\pm$3.5
Lunit-Dino |  80.7$\pm$0.1 |  72.8$\pm$2.1 |  58.7$\pm$3.6 |  41.0$\pm$3.5 |  79.1$\pm$2.1 |  75.1$\pm$1.8 |  64.3$\pm$4.1
Lunit-BT111 |  81.2$\pm$1.3 |  78.7$\pm$0.9 |  74.3$\pm$1.5 |  65.7$\pm$1.9 |  79.6$\pm$0.4 |  78.1$\pm$1.5 |  76.0$\pm$0.8
CTransPath111 |  83.7$\pm$0.6 |  80.4$\pm$1.6 |  77.2$\pm$1.3 |  72.3$\pm$1.0 |  80.4$\pm$0.6 |  79.6$\pm$0.7 |  75.1$\pm$1.2
PathoDuet111 |  79.0$\pm$2.5 |  75.4$\pm$2.1 |  72.7$\pm$2.2 |  68.0$\pm$1.8 |  78.3$\pm$1.0 |  76.4$\pm$0.9 |  71.1$\pm$3.5
iBOT |  79.2$\pm$2.0 |  62.7$\pm$5.2 |  51.2$\pm$4.7 |  34.0$\pm$7.7 |  82.1$\pm$0.7 |  72.3$\pm$1.9 |  56.8$\pm$1.2
MoCo111 |  71.2$\pm$0.5 |  68.8$\pm$1.2 |  65.6$\pm$1.4 |  58.2$\pm$1.5 |  78.2$\pm$0.5 |  76.1$\pm$1.2 |  72.6$\pm$1.4

Method | LC25000 | GasHisSDB
---|---|---
|  $\eta=0$ |  $\eta=0.2$ |  $\eta=0.4$ |  $\eta=0.6$ |  $\eta=0$ |  $\eta=0.2$ |  $\eta=0.4$
Baseline |  100$\pm$0.0 |  95.0$\pm$0.4 |  77.5$\pm$3.7 |  75.8$\pm$4.2 |  98.2$\pm$0.4 |  87.7$\pm$0.2 |  65.5$\pm$2.4
GCE |  99.9$\pm$0.1 |  99.8$\pm$0.1 |  99.4$\pm$0.2 |  96.9$\pm$0.2 |  98.3$\pm$0.1 |  96.0$\pm$0.2 |  74.9$\pm$5.0
APL |  99.9$\pm$0.1 |  99.7$\pm$0.1 |  99.4$\pm$0.1 |  98.1$\pm$0.2 |  98.1$\pm$0.1 |  96.5$\pm$0.1 |  75.4$\pm$0.5
DivideMix |  99.7$\pm$0.2 |  99.6$\pm$0.1 |  99.4$\pm$0.2 |  98.4$\pm$0.1 |  97.9$\pm$0.1 |  97.3$\pm$0.1 |  93.5$\pm$0.4
Phikon |  99.9$\pm$0.1 |  99.6$\pm$0.1 |  99.1$\pm$0.1 |  97.7$\pm$0.5 |  99.6$\pm$0.1 |  86.7$\pm$1.5 |  64.9$\pm$1.6
RetCCL111 |  98.7$\pm$0.2 |  97.8$\pm$0.2 |  97.6$\pm$0.2 |  97.0$\pm$0.2 |  97.2$\pm$0.1 |  95.3$\pm$0.5 |  87.3$\pm$2.5
Lunit-DINO |  99.6$\pm$0.2 |  99.1$\pm$0.1 |  99.0$\pm$0.1 |  97.2$\pm$0.6 |  99.2$\pm$0.1 |  84.9$\pm$2.1 |  65.7$\pm$2.2
Lunit-BT111 |  99.9$\pm$0.1 |  99.6$\pm$0.3 |  99.1$\pm$0.1 |  98.6$\pm$0.1 |  98.5$\pm$0.1 |  97.2$\pm$0.3 |  91.6$\pm$3.2
CTransPath111 |  99.9$\pm$0.1 |  99.3$\pm$0.2 |  99.1$\pm$0.1 |  98.9$\pm$0.1 |  98.7$\pm$0.1 |  97.3$\pm$0.2 |  94.6$\pm$0.3
PathoDuet111 |  99.9$\pm$0.1 |  99.8$\pm$0.1 |  99.6$\pm$0.1 |  98.5$\pm$0.1 |  98.3$\pm$0.1 |  95.9$\pm$0.9 |  92.4$\pm$1.1
iBOT |  99.9$\pm$0.1 |  95.7$\pm$1.3 |  89.2$\pm$0.7 |  67.8$\pm$8.2 |  97.3$\pm$0.4 |  80.1$\pm$1.2 |  60.6$\pm$3.8
MoCo111 |  98.5$\pm$0.1 |  98.2$\pm$0.1 |  97.9$\pm$0.1 |  97.4$\pm$0.2 |  96.2$\pm$0.2 |  91.0$\pm$0.4 |  85.9$\pm$0.6

Figure 1: Average test accuracies (4 runs) of linear classifiers trained with
deep embedding over different label noise ratios. Shaded areas represent
standard deviation. iBOT and MoCo backbones (dashed curves) are pre-trained on
ImageNet.

One might think that these two models extract inferior representations
compared to others. However, we demonstrate that this is not true through a
few-shot learning experiment using k-nearest neighbors (k-NN) classifiers
trained on 10% of each dataset. Idea is to test the inherent resistance of
embeddings and demonstrate that, without proper training of a linear head,
they exhibit uniform reactions to label noise. Results, illustrated in
fig:knn, confirm that hypothesis. Furthermore, upon closer examination through
t-SNE visualization of NCT-CRC-HE-100k in fig:tsne it becomes evident that no
model extracts a worse representation. Thus the observed performance
differences are not due to the quality of the learned representations, but
rather to the noise-resilient property leveraged by the linear classifier when
trained with contrastive embeddings.

Figure 2: Average test accuracies (4 runs) of k-NN classifiers (k=5) trained
on 10% of train datasets over different label noise ratios. Shaded areas
represent standard deviation. Figure 3: t-SNE representations of NCT-CRC-
HE-100k deep embeddings extracted from various histopathology foundation
models.

That explains why in fig:curves, at $\eta=0$, Phikon and Lunit-DINO perform
really well but, at higher noise rates, some contrastive methods start with
lower accuracies than them at $\eta=0$, yet subsequently outpace them, thereby
showcasing their enhanced resilience to label noise. A particularly prominent
example is PatchCamelyon with RetCCL, where the test accuracy starts at 85.8%,
while Phikon and Lunit-DINO start at 86.4% and 88.1%. However, RetCCL
maintains stability as $\eta$ increases, while the other two decline. Under
asymmetric noise, the conclusion is consistent with the previous observations
as shown in tab:asym. Specifically, on the three multiclass datasets, training
with contrastive deep embeddings yields the best results. Moreover, the
results on the two ImageNet backbones under both noise types further support
our findings, with iBOT exhibiting a consistently steeper decline in
performance than MoCo as the noise rate increases.

Table 3: Average test accuracies with standard deviation on NCT-CRC-HE-100k, BACH and LC25000 under different asymetric label noise ratios (%, 2 runs for image methods, 4 runs for deep embedding methods). Class transition matrices are given in appendix C. 111:Foundation model trained with contrastive learning. Method | NCT-CRC-HE-100k | BACH | LC25000
---|---|---|---
$\eta=0.2$ | $\eta=0.4$ | $\eta=0.2$ | $\eta=0.4$ | $\eta=0.2$ | $\eta=0.4$
Baseline |  88.6$\pm$0.9 |  78.4$\pm$1.3 |  60.4$\pm$4.1 |  40.2$\pm$8.4 |  95.1$\pm$1.1 |  69.3$\pm$1.2
GCE |  94.5$\pm$0.3 |  91.3$\pm$1.7 |  65.4$\pm$1.3 |  56.8$\pm$2.7 |  97.2$\pm$0.1 |  92.7$\pm$0.4
APL |  94.5$\pm$0.4 |  92.0$\pm$0.3 |  70.5$\pm$0.9 |  60.7$\pm$2.2 |  97.2$\pm$0.3 |  93.4$\pm$0.5
DivideMix |  96.0$\pm$0.1 |  95.4$\pm$0.2 |  72.4$\pm$1.5 |  61.3$\pm$2.0 |  98.9$\pm$0.1 |  96.5$\pm$0.2
Phikon |  88.0$\pm$4.1 |  74.9$\pm$7.8 |  66.9$\pm$3.5 |  46.7$\pm$3.6 |  98.9$\pm$0.1 |  96.3$\pm$0.7
RetCCL111 |  93.9$\pm 0.4$ |  93.4$\pm$0.3 |  77.4$\pm$1.6 |  61.4$\pm$5.2 |  97.6$\pm$0.1 |  96.8$\pm$0.2
Lunit-DINO |  93.1$\pm$2.6 |  85.7$\pm$6.4 |  65.1$\pm$4.1 |  51.2$\pm$3.9 |  99.1$\pm$0.1 |  97.3$\pm$0.2
Lunit-BT111 |  94.5$\pm$0.2 |  94.2$\pm$0.7 |  69.3$\pm$4.0 |  57.5$\pm$8.4 |  99.5$\pm$0.1 |  97.9$\pm$0.3
CTransPath111 | 97.4$\pm$0.2 | 97.1$\pm$0.3 | 79.0$\pm$1.3 | 66.9$\pm$14.7 |  99.2$\pm$0.3 | 98.5$\pm$0.1
PathoDuet111 |  95.8$\pm$0.3 |  95.6$\pm$0.4 |  73.1$\pm$0.8 |  62.2$\pm$3.5 | 99.6$\pm$0.1 |  98.1$\pm$0.7
iBOT |  82.4$\pm$3.4 |  66.9$\pm$8.7 |  58.6$\pm$4.6 |  45.8$\pm$3.5 |  93.1$\pm$3.3 |  77.5$\pm$3.1
MoCo111 |  92.9$\pm$0.3 |  91.2$\pm$0.4 |  66.8$\pm$0.6 |  62.3$\pm$1.4 |  92.5$\pm$4.2 |  88.3$\pm$2.4

##  5 Limitations

Although the study shows that contrastive learning effectively mitigates the
label noise challenge, it does not completely eliminate it. Specifically, for
relatively small datasets such as BACH and MHIST, the stability over varying
noise rates is lower than the one observed with larger datasets. Similarly,
asymmetric noise appears to be more challenging to resist. Thus, there is
still a need for further research to develop methods that can completely
overcome the label noise issue. Potential avenues for future research include
exploring the combination of contrastive embeddings with other noise
resilience methods such as label cleaning or robust losses or investigating
the impact of fine-tuning the non-contrastive backbones with contrastive
learning. For control over noise rates we used synthetic label noise over
public datasets. A validation study using real-world noisy datasets could
conclude into the effectiveness of contrastive embeddings in handling label
noise.

##  6 Conclusion

Our study highlights the effectiveness of contrastive-based deep embedding
training in bolstering the robustness of histopathology classifiers under
noisy labels. Through extensive experiments across multiple benchmark
datasets, we showcase the superior noise resilience of linear classifiers
trained with contrastive-based deep embeddings when compared to image-based
methods and non-contrastive embeddings. By elucidating these findings, our aim
is to offer valuable insights and guidelines for the future development of
histopathological foundation models. Notably, the majority of current
histology foundation models overlook the assessment of label noise resilience
in their performance evaluations, despite it being one of the most significant
challenges in digital histopathology.

This work was supported by Primaa.

## References

  * Borkowski et al. (2019) Andrew A. Borkowski, Marilyn M. Bui, L. Brannon Thomas, Catherine P. Wilson, Lauren A. DeLand, and Stephen M. Mastorides.  Lung and colon cancer histopathological image dataset (lc25000), 2019.
  * Campanella and Kwan (2023) Gabriele Campanella and Ricky Kwan.  Computational pathology at health system scale – self-supervised foundation models from three billion images, 9 2023.
  * Caron et al. (2020) Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin.  Unsupervised learning of visual features by contrasting cluster assignments.  2020\.
  * Caron et al. (2021) Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin.  Emerging properties in self-supervised vision transformers.  In _Proceedings of the International Conference on Computer Vision (ICCV)_ , 2021.
  * Chen et al. (2023) Richard J. Chen, Tong Ding, Ming Y. Lu, Drew F. K. Williamson, Guillaume Jaume, Bowen Chen, Andrew Zhang, Daniel Shao, Andrew H. Song, Muhammad Shaban, Mane Williams, Anurag Vaidya, Sharifa Sahai, Lukas Oldenburg, Luca L. Weishaupt, Judy J. Wang, Walt Williams, Long Phi Le, Georg Gerber, and Faisal Mahmood.  A general-purpose self-supervised model for computational pathology, 2023.
  * Chen et al. (2020a) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.  A simple framework for contrastive learning of visual representations.  In _Proceedings of the 37th International Conference on Machine Learning_ , ICML’20, 2020a.
  * Chen et al. (2021) X. Chen, S. Xie, and K. He.  An empirical study of training self-supervised vision transformers.  In _2021 IEEE/CVF International Conference on Computer Vision (ICCV)_ , pages 9620–9629, 2021.
  * Chen et al. (2020b) Xinlei Chen, Haoqi Fan, Ross B. Girshick, and Kaiming He.  Improved baselines with momentum contrastive learning.  _ArXiv_ , abs/2003.04297, 2020b.
  * Dawson (2022) Heather Dawson.  Digital pathology – rising to the challenge.  _Frontiers in Medicine_ , page 888896, 2022.
  * Deng et al. (2009) Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE Conference on Computer Vision and Pattern Recognition_ , pages 248–255, 2009.
  * Dippel and Feulner (2024) Jonas Dippel and Barbara Feulner.  Rudolfv: A foundation model by pathologists for pathologists, 0 2024.
  * Ehteshami Bejnordi et al. (2017) Babak Ehteshami Bejnordi, Mitko Veta, Paul Johannes van Diest, Bram van Ginneken, Nico Karssemeijer, Geert Litjens, Jeroen A. W. M. van der Laak, , and the CAMELYON16 Consortium.  Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer.  _JAMA_ , 318(22):2199–2210, 2017.
  * Filiot et al. (2023) Alexandre Filiot, Ridouane Ghermi, Antoine Olivier, Paul Jacob, Lucas Fidon, Alice Mac Kain, Charlie Saillard, and Jean-Baptiste Schiratti.  Scaling self-supervised learning for histopathology with masked image modeling.  _medRxiv_ , 2023.
  * Garcia et al. (2020) Edna Garcia, Iman Kundu, Melissa Kelly, Ryan Soles, Lotte Mulder, and Geoffrey A. Talmon.  The american society for clinical pathology’s job satisfaction, well-being, and burnout survey of pathologists.  _American journal of clinical pathology_ , pages 435–448, 2020.
  * Ghosh et al. (2017) Aritra Ghosh, Himanshu Kumar, and P. Shanti Sastry.  Robust loss functions under label noise for deep neural networks.  In _AAAI Conference on Artificial Intelligence_ , 2017.
  * Han et al. (2018) Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor W. Tsang, and Masashi Sugiyama.  Co-teaching: Robust training of deep neural networks with extremely noisy labels.  In _Proceedings of the 32nd International Conference on Neural Information Processing Systems_ , NIPS’18, page 8536–8546, 2018.
  * He et al. (2015) Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 770–778, 2015.
  * He et al. (2020) Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick.  Momentum contrast for unsupervised visual representation learning.  In _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2020.
  * Hu et al. (2022) Weiming Hu, Chen Li, Xiaoyan Li, Md Mamunur Rahaman, Jiquan Ma, Yong Zhang, Haoyuan Chen, Wanli Liu, Changhao Sun, Yudong Yao, Hongzan Sun, and Marcin Grzegorzek.  Gashissdb: A new gastric histopathology image dataset for computer aided diagnosis of gastric cancer.  _Computers in Biology and Medicine_ , page 105207, 2022.
  * Hua et al. (2023) Shengyi Hua, Fang Yan, Tianle Shen, and Xiaofan Zhang.  Pathoduet: Foundation models for pathological slide analysis of h&e and ihc stains, 2023.
  * Jiang et al. (2023) Hongyang Jiang, Mengdi Gao, Yan Hu, Qi Ren, Zhaoheng Xie, and Jiang Liu.  Label-noise-tolerant medical image classification via self-attention and self-supervised learning.  _arXiv preprint arXiv:12306.09718_ , 2023.
  * Kang et al. (2023) Mingu Kang, Heon Song, Seonwook Park, Donggeun Yoo, and Sérgio Pereira.  Benchmarking self-supervised learning on diverse pathology datasets.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 3344–3354, 2023.
  * Kather et al. (2018) Jakob Nikolas Kather, Niels Halama, and Alexander Marx.  100,000 histological images of human colorectal cancer and healthy tissue, May 2018.
  * Khanal et al. (2023) Bidur Khanal, Binod Bhattarai, Bishesh Khanal, and Cristian A. Linte.  Improving medical image classification in noisy labels using only self-supervised pretraining.  In _DEMI@MICCAI_ , 2023.
  * Kim et al. (2021) Yoo Jung Kim, Hyungjoon Jang, Kyoungbun Lee, Seongkeun Park, Sung-Gyu Min, Choyeon Hong, Jeong Hwan Park, Kanggeun Lee, Jisoo Kim, Wonjae Hong, Hyun Jung, Yanling Liu, Haran Rajkumar, Mahendra Khened, Ganapathy Krishnamurthi, Sen Yang, Xiyue Wang, Chang Hee Han, Jin Tae Kwak, Jianqiang Ma, Zhe Tang, Bahram Marami, Jack Zeineh, Zixu Zhao, Pheng-Ann Heng, Rüdiger Schmitz, Frederic Madesta, Thomas Rösch, Rene Werner, Jie Tian, Elodie Puybareau, Matteo Bovio, Xiufeng Zhang, Yifeng Zhu, Se Young Chun, Won-Ki Jeong, Peom Park, and Jinwook Choi.  Paip 2019: Liver cancer segmentation challenge.  _Medical Image Analysis_ , 67:101854, 2021.
  * Kolesnikov et al. (2021) Alexander Kolesnikov, Alexey Dosovitskiy, Dirk Weissenborn, Georg Heigold, Jakob Uszkoreit, Lucas Beyer, Matthias Minderer, Mostafa Dehghani, Neil Houlsby, Sylvain Gelly, Thomas Unterthiner, and Xiaohua Zhai.  An image is worth 16x16 words: Transformers for image recognition at scale.  2021\.
  * Kurian et al. (2022) Nikhil Cherian Kurian, S Varsha, Akshay Bajpai, Sunil Patel, and Amit Sethi.  Improved histology image classification under label noise via feature aggregating memory banks.  In _2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI)_ , 2022.
  * Lee et al. (2017) Kuang-Huei Lee, Xiaodong He, Lei Zhang, and Linjun Yang.  Cleannet: Transfer learning for scalable image classifier training with label noise.  _2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 5447–5456, 2017.
  * Li et al. (2018) Junnan Li, Yongkang Wong, Qi Zhao, and M. Kankanhalli.  Learning to learn from noisy labeled data.  _2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 5046–5054, 2018.
  * Li et al. (2020) Junnan Li, Richard Socher, and Steven C. H. Hoi.  Dividemix: Learning with noisy labels as semi-supervised learning.  _ICLR 2020_ , 2020.
  * Liu et al. (2021) Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo.  Swin transformer: Hierarchical vision transformer using shifted windows.  In _2021 IEEE/CVF International Conference on Computer Vision (ICCV)_ , pages 9992–10002, 2021.
  * Loshchilov and Hutter (2017) Ilya Loshchilov and Frank Hutter.  SGDR: stochastic gradient descent with warm restarts.  In _5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings_ , 2017.
  * Lu et al. (2023) Ming Y. Lu, Bowen Chen, Drew F. K. Williamson, Richard J. Chen, Ivy Liang, Tong Ding, Guillaume Jaume, Igor Odintsov, Andrew Zhang, Long Phi Le, Georg K. Gerber, Anil Parwani, and Faisal Mahmood.  Towards a visual-language foundation model for computational pathology.  _ArXiv_ , abs/2307.12914, 2023.
  * Ma et al. (2020) Xingjun Ma, Hanxun Huang, Yisen Wang, Simone Romano, Sarah M. Erfani, and James Bailey.  Normalized loss functions for deep learning with noisy labels.  _ICML2020_ , 2020.
  * Northcutt et al. (2017) Curtis G. Northcutt, Tailin Wu, and Isaac L. Chuang.  Learning with confident examples: Rank pruning for robust classification with noisy labels.  In _Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intelligence_ , UAI’17, 2017.
  * Polónia et al. (2020) António Polónia, Catarina Eloy, and Paulo Aguiar.  BACH Dataset : Grand Challenge on Breast Cancer Histology images, January 2020.
  * Robbins (1951) Herbert E. Robbins.  A stochastic approximation method.  _Annals of Mathematical Statistics_ , 22:400–407, 1951.
  * Tan et al. (2021) Cheng Tan, Jun Xia, Lirong Wu, and Stan Z. Li.  Co-learning: Learning from noisy labels with self-supervision.  _Proceedings of the 29th ACM International Conference on Multimedia_ , 2021.
  * Veeling et al. (2018) Bastiaan S Veeling, Jasper Linmans, Jim Winkens, Taco Cohen, and Max Welling.  Rotation equivariant CNNs for digital pathology.  June 2018.
  * Vorontsov et al. (2023) Eugene Vorontsov, Alican Bozkurt, Adam Casson, George Shaikovski, Michal Zelechowski, Siqi Liu, Kristen Severson, Eric Zimmermann, James Hall, Neil Tenenholtz, Nicolo Fusi, Philippe Mathieu, Alexander van Eck, Donghun Lee, Julian Viret, Eric Robert, Yi Kan Wang, Jeremy D. Kunz, Matthew C. H. Lee, Jan Bernhard, Ran A. Godrich, Gerard Oakley, Ewan Millar, Matthew Hanna, Juan Retamero, William A. Moye, Razik Yousfi, Christopher Kanan, David Klimstra, Brandon Rothrock, and Thomas J. Fuchs.  Virchow: A Million-Slide Digital Pathology Foundation Model.  _arXiv e-prints_ , 2023.
  * Wang et al. (2021) Xiyue Wang, Sen Yang, Jun Zhang, Minghui Wang, Jing Zhang, Junzhou Huang, Wei Yang, and Xiao Han.  Transpath: Transformer-based self-supervised learning for histopathological image classification.  In _International Conference on Medical Image Computing and Computer-Assisted Intervention_ , pages 186–195. Springer, 2021.
  * Wang et al. (2022) Xiyue Wang, Sen Yang, Jun Zhang, Minghui Wang, Jing Zhang, Wei Yang, Junzhou Huang, and Xiao Han.  Transformer-based unsupervised contrastive learning for histopathological image classification.  _Medical Image Analysis_ , 81:102559, 2022.
  * Wang et al. (2023) Xiyue Wang, Yuexi Du, Sen Yang, Jun Zhang, Minghui Wang, Jing Zhang, Wei Yang, Junzhou Huang, and Xiao Han.  Retccl: Clustering-guided contrastive learning for whole-slide image retrieval.  _Medical Image Analysis_ , 83:102645, 2023.
  * Wei et al. (2021) Jerry Wei, Arief Suriawinata, Bing Ren, Xiaoying Liu, Mikhail Lisovsky, Louis Vaickus, Charles Brown, Michael Baker, Naofumi Tomita, Lorenzo Torresani, et al.  A petri dish for histopathology image analysis.  In _International Conference on Artificial Intelligence in Medicine_ , pages 11–24. Springer, 2021.
  * Weinstein et al. (2013) John N Weinstein, Eric A Collison, Gordon B. Mills, Kenna M Shaw, Kyle Elrott, Ilya Shmulevich, Chris Sander, and Joshua M Stuart.  The cancer genome atlas pan-cancer analysis project.  In _Nature genetics_ , page 1113–1120, 2013.
  * Xue et al. (2022) Yihao Xue, Kyle Whitecross, and Baharan Mirzasoleiman.  Investigating why contrastive learning benefits robustness against label noise.  In _Proceedings of the 39th International Conference on Machine Learning_ , volume 162, 2022.
  * Yu et al. (2019) Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor Wai-Hung Tsang, and Masashi Sugiyama.  How does disagreement help generalization against label corruption?  In _International Conference on Machine Learning_ , 2019.
  * Zbontar et al. (2021) Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny.  Barlow twins: Self-supervised learning via redundancy reduction.  In _Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event_ , Proceedings of Machine Learning Research, 2021.
  * Zhang et al. (2016) Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals.  Understanding deep learning requires rethinking generalization.  _arXiv preprint arXiv:1611.03530_ , 2016.
  * Zhang and Sabuncu (2018) Zhilu Zhang and Mert R. Sabuncu.  Generalized cross entropy loss for training deep neural networks with noisy labels.  In _Proceedings of the 32nd International Conference on Neural Information Processing Systems_ , NIPS’18, page 8792–8802, 2018.
  * Zhou et al. (2022) Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan Yuille, and Tao Kong.  ibot: Image bert pre-training with online tokenizer.  _International Conference on Learning Representations (ICLR)_ , 2022.
  * Zhu et al. (2022) Chuang Zhu, Wenkai Chen, Ting Peng, Ying Wang, and Mulan Jin.  Hard sample aware noise robust learning for histopathology image classification.  _IEEE transactions on medical imaging_ , 2022.

##  Appendix A Experimental Settings

As a baseline, we use a ResNet-50 model trained with the CCE loss starting
from ImageNet Deng et al. (2009) weights. For baseline runs we used a
stochastic gradient descent (SGD) optimizer Robbins (1951) with learning rate
of 0.0005 coupled to cosine annealing scheduler Loshchilov and Hutter (2017)
and early stopping with 20 patience epochs. For GCE, we set $q$ to $0.7$ and
used the same settings as baseline. For APL, we used the normalized cross
entropy - reverse cross entropy (NCE-RCE) loss with $\alpha=0.6$ and
$\beta=0.4$. For both GCE and APL we did a five epochs warmup with CCE loss.
For DivideMix, we used default settings and a learning rate of 0.001 for 50
epochs. For embedding-based trainings, a four-layer linear neural network
serves as the classifier model. We used grid search to find out optimal
learning rate, batch size, patience and Gaussian noise $\sigma$ for each
backbone-dataset combinations. All runs are conducted on Nvidia GeForce RTX
2080 Ti GPUs.

##  Appendix B Datasets Details

NCT-CRC-HE-100K consists of 100,000 hematoxylin and eosin (H&E) stained
224x224 histological images of human colorectal cancer (CRC) divided in 9
tissue classes (adipose, background, debris, lymphocytes, mucus, smooth
muscle, normal colon mucosa, cancer-associated stroma, colorectal
adenocarcinoma epithelium). The test set uses a different CRC-VAL-HE-7K
dataset consisting of 7,180 patches featuring all nine classes. PatchCamelyon
binary dataset contains 327,680 stained 96x96 patches derived from
histopathological scans of lymph node sections with positive label indicating
presence of metastatic tissue. It provides train, validation and test sets.
PatchCamelyon is derived from the Camelyon16 challenge Ehteshami Bejnordi et
al. (2017). BACH dataset is composed of H&E stained breast histology
microscopy 2048x1536 patches. Train and test sets respectively possess 400 and
83 patches distributed in 4 classes according to the predominant cancer type
in each image (normal, benign, in situ carcinoma, invasive carcinoma). As the
ground truth was not available for the test set, we asked a breast pathologist
expert to label the 100 patches. Among the 100, 17 were considered uncertain
due to a lack of contextual information on the patch so we chose to not use
these 17 patches in order to have a clean ground truth. MHIST is made up of
3,152 H&E stained 224x224 images of colorectal polyps split into 2175 and 977
for train/test sets. Binary classes, hyperplastic polyp (HP) or sessile
serrated adenoma (SSA), indicate the predominant histological pattern. LC25000
dataset contains five classes of 5,000 768x768 images of lung and colon
tissues. The classes are: colon adenocarcinomas, benign colonic tissues, lung
adenocarcinomas, lung squamous cell carcinomas and benign lung tissues.
GasHisSDB consists of 32,284 160x160 H&E stained gastric cancer pathology
patches. Classes are abnormal and normal. In these two last datasets there is
no test set provided so we split ourselves train set into 80:20 for train/test
sets then 80:20 for train/validation sets.

For image training methods, we employ a traditional preprocessing and
augmentation pipeline. For the training set, we incorporate resizing,
horizontal and vertical flipping, color jittering, and normalization. For the
validation and test sets, we simplify the process by only resizing and
normalizing the images. The resizing dimensions are set to 512x512 for BACH
and LC25000 datasets, while for the remaining datasets, we maintain the
original sizes.

For embedding extractions, we consider the fact that backbones require an
input size of 224x224. Therefore, for oversized images, such as those in the
BACH and LC25000 datasets, we resize them to this resolution. For undersized
images, like those in the PatchCamelyon and GasHisSDB datasets, we experiment
with both resizing and padding. Our results indicate that resizing yields
better outcomes. The size of the embeddings varies depending on the backbone
used. For Phikon, CTransPath, and PathoDuet, the embedding size is 768, while
for RetCCL and Lunit-BT, it is 2048. Lunit-DINO, on the other hand, uses an
embedding size of 384.

For training methods based on embeddings, we introduce an additional
augmentation to the training set by adding random Gaussian noise, scaled with
$\sigma$.

##  Appendix C Asymetric Noise Transition Matrices

Table 4: NCT-CRC-HE-100k class transition matrix. | ADI | BACK | DEB | LYM | MUC | MUS | NORM | STR | TUM
---|---|---|---|---|---|---|---|---|---
ADI | $1-\eta$ | $0$ | $0$ | $0$ | $0$ | $\frac{\eta}{2}$ | $\frac{\eta}{2}$ | $0$ | 0
BACK | $0$ | $1-\eta$ | $\frac{\eta}{3}$ | $\frac{\eta}{3}$ | $\frac{\eta}{3}$ | $0$ | $0$ | $0$ | 0
DEB | $0$ | $\frac{\eta}{3}$ | $1-\eta$ | $\frac{\eta}{3}$ | $\frac{\eta}{3}$ | $0$ | $0$ | $0$ | 0
LYM | $0$ | $\frac{\eta}{3}$ | $\frac{\eta}{3}$ | $1-\eta$ | $\frac{\eta}{3}$ | $0$ | $0$ | $0$ | 0
MUC | $0$ | $\frac{\eta}{3}$ | $\frac{\eta}{3}$ | $\frac{\eta}{3}$ | $1-\eta$ | $0$ | $0$ | $0$ | 0
MUS | $\frac{\eta}{2}$ | $0$ | $0$ | $0$ | $0$ | $1-\eta$ | $\frac{\eta}{2}$ | $0$ | 0
NORM | $\frac{\eta}{2}$ | $0$ | $0$ | $0$ | $0$ | $\frac{\eta}{2}$ | $1-\eta$ | $0$ | 0
STR | $0$ | $0$ | $0$ | $0$ | $0$ | $0$ | 0 | $1-\eta$ | $\eta$
TUM | 0 | $0$ | $0$ | $0$ | $0$ | $0$ | $0$ | $\eta$ | $1-\eta$
Table 5: BACH class transition matrix. | Benign | CIS | CI | Normal
---|---|---|---|---
Benign | $1-\eta$ | $\eta$ | $0$ | $0$
CIS | $\frac{\eta}{2}$ | $1-\eta$ | $\frac{\eta}{2}$ | $0$
CI | $0$ | $\frac{\eta}{2}$ | $1-\eta$ | $\frac{\eta}{2}$
Normal | $0$ | $0$ | $\eta$ | $1-\eta$
Table 6: LC25000 class transition matrix. | Colon ACA | Benign Colon | Lung ACA | Benign Lung | Lung SCC
---|---|---|---|---|---
Colon ACA | $1-\eta$ | $\frac{\eta}{2}$ | $\frac{\eta}{2}$ | $0$ | $0$
Benign Colon | $\frac{\eta}{2}$ | $1-\eta$ | $0$ | $\frac{\eta}{2}$ | $0$
Lung ACA | $\frac{\eta}{2}$ | $0$ | $1-\eta$ | $0$ | $\frac{\eta}{2}$
Benign Lung | 0 | $\eta$ | $0$ | $1-\eta$ | $0$
Lung SCC | $0$ | $0$ | $\eta$ | $0$ | $1-\eta$