# LEFM-Nets: Learnable Explicit Feature Map Deep Networks for Segmentation of
Histopathological Images of Frozen Sections

Dario Sitnik and Ivica Kopriva* Senior Member  IEEE

###### Abstract

Accurate segmentation of medical images is essential for diagnosis and
treatment of diseases. These problems are solved by highly complex models,
such as deep networks (DN), requiring a large amount of labeled data for
training. Thereby, many DNs possess task- or imaging modality specific
architectures with a decision-making process that is often hard to explain and
interpret. Here, we propose a framework that embeds existing DNs into a low-
dimensional subspace induced by the learnable explicit feature map (LEFM)
layer. Compared to the existing DN, the framework adds one hyperparameter and
only modestly increase the number of learnable parameters. The method is aimed
at, but not limited to, segmentation of low-dimensional medical images, such
as color histopathological images of stained frozen sections. Since features
in the LEFM layer are polynomial functions of the original features, proposed
LEFM-Nets contribute to the interpretability of network decisions. In this
work, we combined LEFM with the known networks: DeepLabv3+, UNet, UNet++ and
MA-net. New LEFM-Nets are applied to the segmentation of adenocarcinoma of a
colon in a liver from images of hematoxylin and eosin (H&E) stained frozen
sections. LEFM-Nets are also tested on nuclei segmentation from images of H&E
stained frozen sections of ten human organs. On the first problem, LEFM-Nets
achieved statistically significant performance improvement in terms of micro
balanced accuracy and $\mathbf{F}_{1}$ score than original networks. When
averaged over ten runs, LEFM-MA-net achieved balanced accuracy of
$\mathbf{89.36\boldsymbol{\%}\boldsymbol{\pm}1.28\boldsymbol{\%}}$ compared to
$\mathbf{88.02\boldsymbol{\%}\boldsymbol{\pm}1.22\boldsymbol{\%}}$ by the MA-
net. Corresponding results for $\mathbf{F}_{1}$ score are
$\mathbf{84.96\boldsymbol{\%}\boldsymbol{\pm}1.14\boldsymbol{\%}}$ and
$\mathbf{82.75\boldsymbol{\%}\boldsymbol{\pm}1.10\boldsymbol{\%}}$. On the
second problem, LEFM-Nets achieved only better performance in comparison with
the original networks. LEFM-MA-net achieved balanced accuracy of
$\mathbf{89.41\boldsymbol{\%}\boldsymbol{\pm}0.29\boldsymbol{\%}}$ compared to
$\mathbf{89.30\boldsymbol{\%}\boldsymbol{\pm}0.44\boldsymbol{\%}}$ by the
original MA-net. Results for $\mathbf{F}_{1}$ score are
$\mathbf{85.35\boldsymbol{\%}\boldsymbol{\pm}0.25\%}$ and
$\mathbf{85.12\%\boldsymbol{\pm}0.51\boldsymbol{\%}}$. The source code is
available at https://github.com/dsitnik/lefm

††This work has been supported in part by the Croatian Science Foundation
Grant IP-2016-06-5235, and in part by the European Regional Development Fund
under the grant KK.01.1.1.01.0009 (DATACROSS).††Dario Sitnik is with the
Division of Electronics, Ruđer Bošković Institute, Bijenička cesta 54, Zagreb,
Croatia. e-mail<EMAIL_ADDRESS>Kopriva is with the Division of
Electronics, Ruđer Bošković Institute, Bijenička cesta 54, Zagreb, Croatia.
e-mail<EMAIL_ADDRESS>Corresponding author

LEFM, histopathological image segmentation, frozen sections, interpretability,
adenocarcinoma of a colon in a liver, nuclei segmentation

##  1 Introduction

Segmentation and/or classification of medical images plays a vital role in
disease diagnosis, therapy planning and follow-up tracking of therapy
efficiency. Therefore, segmentation/classification by using deep learning (DL)
models, i.e. deep neural networks (NNs), is steadily in focus in biomedical
image analysis [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], to cite a few. Thus, a
constant need for performance improvement results in many specialized and
extremely complex DL models, whereat some of them are cited above. Their
decision-making process is often hard to explain and sometimes even harder to
interpret in terms of features understandable to medical experts [11, 12].
When high-stakes judgments must be made, it is critical to comprehend the
reasons behind the decision-making process. As a result, rather than
explaining black box models, it is preferable to create interpretable models
[11]. Furthermore, there is currently a scarcity of expert-annotated datasets
in many medical imaging modalities. For sophisticated models like
convolutional NNs (CNNs), this causes a generalization challenge [2, 13, 14].
This issue is also present in histopathological image segmentation of
intraoperatively collected frozen sections [15]. One reason is that sample
collection during surgery is distracting. Also, another reason is highly time-
consuming effort for pathologists to label cancerous pixels [16].

Figure 1: Proposed framework comprising the slightly modified existing deep
networks such as UNet, UNet++, DeepLabv3+, and MA-net, embedded in a space
induced by the learnable explicit feature map (LEFM) layer. Increased number
of learnable weights in the LEFM layer is represented as $\mathbf{a}_{m}$.
Additional deep network’s weights are introduced through modification to
accept expanded features. $\phi_{m}(\mathbf{X})$ is defined as Hadamard
product ($\otimes$) of: $\psi_{m}(\mathbf{x_{t}})$ \- the spatial location
dependent map of monomials of the input data space features, and
$\mathbf{a}_{m}$ \- the spatial location invariant vector of coefficients. The
first part contributes prior knowledge in terms of monomial-based predefined
algebraic structure. The second part learns the importance of explicit
features.

The issues outlined above motivated us to propose an inclusive framework for
DL-based segmentation/classification of medical images in general, and color
histopathological images of frozen sections in particular. Instead of
designing specialized DL models, we aim to improve segmentation/classification
performance of the existing NNs, such as UNet [17], UNet++ [18, 19],
DeepLabv3+ [20] and MA-net [21]. To achieve that goal, we embed the existing
network in a space induced by the learnable explicit feature map (LEFM) layer
(Fig 1). The LEFM layer is an analog to learnable kernels [22, 23, 24, 25, 26,
27, 28, 29], and its use is motivated by the need to create low-dimensional
subspace in data-adaptable Hilbert space where existing network will yield
improved performance. To reduce risk of overfitting, the aim was to only
modestly increase the complexity relative to the original network. That is
achieved by using the LEFM of low order, whereat the LEFM order is the only
new hyperparameter introduced by the proposed framework. Its suggested value
is either 2 or 3, which simplifies the cross-validation procedure.

The explanation why the LEFM layer improves performance of deep network is in
representation of the LEFM. The LEFM is represented as Hadamard (entry-wise)
product of the spatial location invariant vector of coefficients and spatial
location dependent map composed of the monomials of features from the input
data space. Thus, the predefined (known) algebraic structure of the EFM
contains a priori knowledge that contributes to improved performance of the
proposed framework. For example, a priori knowledge in a form of translation
symmetry in images is incorporated in CNNs [30], and enables them to achieve
translation equivariance by re-using convolutional filters at all spatial
locations. Such design highly reduces the number of learnable parameters in
CNNs. Analogously, the learned vector of coefficients, in proposed LEFM, can
be re-used at all spatial locations with location dependent map of monomials.
Learnable EFM alleviates the analogous problem in kernel-based nonlinear
methods: the choice of kernel. After learning, the coefficients emphasize the
importance of the features in LEFM. Since all features are known polynomial
functions of the original features from the input data space, that contributes
to explainability and interpretability of the decision-making process of the
proposed framework. The main contributions of this work are listed as follows:

  * •

Framework for DL-based segmentation/classification of low-dimensional medical
images comprising slightly modified existing deep network embedded into a
subspace induced by learnable EFM layer.

  * •

Representation of learnable EFM in terms of spatial location dependent map of
monomials of features from the input data space and spatial location invariant
vector of coefficients. The first part brings prior knowledge in terms of
predefined algebraic structure with monomial-based explicit features. The
second part learns the importance of explicit features. That contributes to
medically explainable and interpretabe decision-making. The overall
representation modestly increases the number of learnable parameters compared
to the original network.

  * •

Extensive validation of the proposed framework where we used existing deep
architectures: DeepLabv3+, UNet, UNet++, and MA-net on two problems. The first
one was image segmentation of adenocarcinoma of a colon in a liver from
hematoxylin and eosin (H&E) stained frozen sections. The second problem was
the segmentation of nuclei from H&E stained frozen section images of ten human
organs. Proposed framework was compared to the original networks. In the first
problem, statistically significant improvements in terms of micro balanced
accuracy, micro $F_{1}$ score and micro precision are obtained. In the second,
the proposed framework yielded only better results.

##  2 Related work

###  2.1 Brief overview of some specialized deep neural networks

Herein, we briefly comment on recently proposed deep networks with task- or
imaging modality specific architectures with an emphasis on histopathological
image analysis. New deep CNN was proposed in [4] for medical image
segmentation. It exploits specific attributes in the input datasets. That is
achieved with auxiliary supervision implemented in terms of object perceptive
field and layer-wise effective receptive fields. Comprehensive attention-based
CNN (CA-Net) is proposed in [5] for more accurate and more explainable medical
image segmentation. That is achieved by using multiple attention modules in
CNN architecture aware of the most important spatial positions, channels and
scales[5]. Although the explainability of decision is improved, the
interpretability of the features is still problematic. AttentionBoost network
is proposed in [6] for gland instance segmentation in histopathological
images. It is based on a multi-attention learning model that, based on
adaptive boosting, adjusts loss of fully convolutional networks to adaptively
learn what to attend at each stage. In [25], a new methodology, named PathAL,
is presented aiming to improve classification performance in histopathology
image analysis with a reduced number of expert annotations. Guided soft
attention network for classification of histopathological breast cancer images
is proposed in [8]. In particular, region-level supervision is used to guide
attention of CNN. The attention mechanism activates neurons in diagnostically
relevant regions and suppresses activation in diagnostically irrelevant
regions. Hard sample aware noise robust learning deep network for
classification of histopathology images is proposed in [9]. The method is
aimed at distinguishing informative hard samples from the harmful noisy ones.
That lowers the noise rate and yields an almost clean dataset for training
network in a self-supervised manner. Novel ThresholdNet [10] is designed for
automatic segmentation of polyp in endoscopy images such that segmentation
threshold is learnable. Proposed LEFM-Net framework is directly applicable to
methods proposed in [4, 5, 6, 7, 8, 19, 10]. In this paper, however, we
demonstrated performance improvement of the LEFM-Net framework using well know
deep networks: DeepLabv3+, UNet, UNet++ and MA-net.

###  2.2 Brief overview of kernel-based learning

Let us assume there is a function $f:=\mathbb{R}^{d}\rightarrow\mathbb{R}$
that assigns the class label $y\in S_{C}$ to the sample
$\mathbf{x}\in\mathbb{R}^{d}$. Here, $d$ stands for dimension of the ambient
input space, $C$ stands for the number of classes and $S_{C}$ is a set
containing all possible labels. As an example, for binary classification
problem we have $C=2$ and $S_{2}=\\{0,1\\}$ or $S_{2}=\\{-1,1\\}$. The
classifier learning problem is to estimate $f$ from the given set of training
data $\left\\{\left(\mathbf{x}_{t},y_{t}\right)\right\\}_{t=1}^{T}$. The
representer theorem [31, 32], states that each function $f$ in Hilbert space
$\mathcal{H}$, a.k.a. feature space, that minimizes an arbitrary loss
function, admits representation:

$f(\mathbf{x})=\sum_{t=1}^{T}\alpha_{t}\kappa\left(\mathbf{x}_{t},\mathbf{x}\right)$
(1)

where $\\{\alpha_{t}\\}_{t=1}^{T}$, are estimated from the training data. That
gave rise to the development of many machine learning algorithms, such as
support vector machines and kernel principal component analysis, to name a
few. These methods work well when the underlying problem fits $\mathcal{H}$
properly, i.e. when the given empirical values stem from $f\in\mathcal{H}$. In
other words, the kernel function $\kappa(\circ,\circ)$ has to be chosen
properly. If the appropriate $\mathcal{H}$ is not known, one can resort to
data adaptive kernel methods [23, 24, 25], multiple kernel learning methods
[26], multi-layer (deep) multiple kernel learning method [27], and deep kernel
method [28]. In particular, the last two approaches combine flexibility of
deep networks. One where feature detection is done automatically with the
approximation power of kernel methods. The second one is where the feature map
is determined implicitly by the kernel function. Moreover, in [29] the
representer theorem is proven for a multilayer network of $L$ concatenated
kernel functions. The power of kernel methods is their capability to
implicitly learn potentially infinite dimensional nonlinear features in the
reproducing kernel Hilbert space $\mathcal{H}$. The computation is made
tractable by leveraging the kernel trick [29]. However, implicitly learned
features are hard to interpret.

##  3 Learnable feature maps and deep networks

As pointed out in [28], one of the main reasons to exploit deep kernel
learning was growing frustration among researchers with too many design
options associated with deep networks. Furthermore, the decision-making
process of deep networks is also difficult to interpret. However, deep
networks composed of multiple layers of nonlinear functions can approximate a
rich set of naturally occurring input-output dependencies [22]. Thus, they can
achieve high classification performance. Therefore, we aim to propose a
framework that further improves the performance of deep networks, contributes
to the interpretability of the results and brings a modest increase in the
number of learnable parameters compared to original networks. Towards that
goal, we propose to embed existing deep networks in a subspace induced by the
LEFM of small order (typically 2 or 3). Our approach relies on the well know
kernel trick [31]: for any positive definite kernel function
$\kappa(\mathbf{x},\mathbf{y})$ there exists a function $\phi(\mathbf{x})$
that maps data $\mathbf{x}$ to a Hilbert space $\mathcal{H}$, such that
$\kappa(\mathbf{x},\mathbf{y})=\langle\phi(\mathbf{x}),\phi(\mathbf{y})\rangle_{\mathcal{H}}\
\cdot\ \phi(\mathbf{x})$ is known as EFM. Thus, we can recast the representer
theorem (1) in terms of kernel associated EFM:

$f(\mathbf{x})=\langle\mathbf{w},\phi(\mathbf{x})\rangle_{\mathcal{H}}$ (2)

where $\mathbf{w}=\sum_{t=1}^{T}\alpha_{t}\phi\left(\mathbf{x}_{t}\right)$.
Thus, the nonlinear approximation problem (1) in the input data space becomes
the linear one in the feature space $\mathcal{H}$. Hence, kernel-based
nonlinear algorithms can be seen as linear algorithms operating in appropriate
feature space [33]. That is formalized by the Cover’s theorem [34]. It states
that the number of separating hyperplanes is proportional to the dimension of
the feature space. Thus, it is important for the dimension of expanded feature
space $D$ to satisfy $D>>d$. That is fulfilled easily since EFMs associated
with many popular kernels are mostly high dimensional or even infinite
dimensional (e.g. EFM associated with the Gaussian kernel). That is also a
reason feature maps are not used explicitly in nonlinear algorithms [33].
Nevertheless, we can partially circumvent this obstacle by using the
approximate EFM of order $m$, i.e. $\phi_{m}(\mathbf{x})\in\mathbb{R}^{D}$. In
that regard, we emphasize that EFMs can be written in a common algebraic form:

$\phi_{m}(\mathbf{x})=\left[\left\\{a_{q_{1},\ldots,q_{d}}x_{1}^{q_{1}}\ldots
x_{d}^{q_{d}}\right\\}_{q_{1},\ldots,q_{d}=0}^{m}\right]^{\mathrm{T}}$ (3)

$\text{such that}\sum_{i=1}^{d}q_{i}\leq m.$

When $m\rightarrow\infty$ we have:
$\phi_{m}(\mathbf{x})\rightarrow\phi(\mathbf{x})$. Bearing in mind that the
dimension of induced feature space $D$ depends on $d$ and $m$ through:

$D=\binom{d+m}{m}$ (4)

it becomes clear that we need to keep $m$ low if we do not want to introduce
too many learnable parameters in the proposed framework. Thus, in the
experiments conducted in this paper, we have $m\in\\{2,3\\}$. Hence, the
formulation (2) with approximate EFM $\phi_{m}(\mathbf{x})$ will only
approximate the original nonlinear approximation problem (1). That is why we
need to embed the deep network in a subspace
$\mathcal{H}_{D}\subset\mathcal{H}$ induced by $\phi_{m}(\mathbf{x})$. By
doing so, we expect to further boost segmentation/classification performance
of the network. Nevertheless, the problem related to the proper fit of the
underlying problem to $\mathcal{H}$ still exists. Thus, we propose the
following computationally efficient representation of $\phi_{m}(\mathbf{x})$:

$\phi_{m}(\mathbf{x}):=\psi_{m}(\mathbf{x})\otimes\mathbf{a}_{m}$ (5)

where $\otimes$ denotes Hadamard (entry wise) product, and:

$\psi_{m}(\mathbf{x}):=\left[\left\\{x_{1}^{q_{1}}\ldots
x_{d}^{q_{d}}\right\\}_{q_{1},\ldots,q_{d}=0}^{m}\right]^{\mathrm{T}}\text{s.t.}\sum_{i=1}^{d}q_{i}\leq
m,$ (6)

$\mathbf{a}_{m}:=\left[\left\\{a_{q_{1},\ldots,q_{d}}\right\\}_{q_{1},\ldots,q_{d}=0}^{m}\right]^{\mathrm{T}}\text{s.t.}\sum_{i=1}^{d}q_{i}\leq
m.$ (7)

Hence, when existing deep network is trained with $\phi_{m}(\mathbf{x})$, the
coefficients $\mathbf{a}_{m}$ are learned together with the rest of network
parameters. Since $\mathbf{a}_{m}$ is spatial location invariant, learning is
computationally efficient. As opposed to that, $\psi_{m}(\mathbf{x})$ is a
spatial location dependent function of the input features and it is not
learnable. Thus, when $d$ and $m$ are small, the number $D$ of learnable
coefficients in $\mathbf{a}_{m}$ is modest when compared to the millions of
parameters in existing deep networks. Number of trainable parameteres is
additionally increased in amounts specific to the deep architecture and its
backbone. Thus, all newly introduced parameters are directly related to the
increase from $d$ to $D$ dimensional input. However, vast majority of the
original network remains the same.

As an example, in the problem considered in the paper, we apply the proposed
framework to segmentation of color (RGB) histopathological images of H&E
stained frozen sections. Thus, we have $d=3$. As we worked with approximate
EFM of order $m\in\\{2,3\\}$, that respectively yields $D\in\\{10,20\\}$. The
rationale why the deep network, embedded in a subspace
$\mathcal{H}_{D}\subset\mathcal{H}$ induced by learnable EFM, outperforms the
original network operating in the input data space is due to the explicit
character of features up to the selected order $m$. Through learning, a deep
network can determine their importance. Thus, in the proposed framework, the
network will use more explicit features than it is available in the input data
space. Since features in $\phi_{m}(\mathbf{x})$ are polynomials comprising the
originals features from the input data space, it is possible to interpret
them.

##  4 Experiments and Results

###  4.1 Data adaptive feature expansion layer

For implementing an LEFM layer, fundamental functionality is described in (5)
- (7). As mentioned, the coefficients are data adaptive (i.e. learnable) and,
for LEFM layer construction, we are only interested in monomials. Thus, we
need to find monomials (6) using all combinations of $\boldsymbol{q}$ such
that $\sum_{i=1}^{d}{q_{i}}\leq m,\ \forall q_{i}\geq 0$. Thus, the number of
terms in this setting becomes (4). Optionally, if the LEFM layer is followed
by batch normalization layer [35], two parameters per each term (mean and
standard deviation) are introduced.

The core functionality of LEFM layer is developed using two matrices, term-
and power mask. Both matrices are $D\times d$ dimensional. The term mask
matrix $\mathbf{TM}$ is associated with selecting features
$\\{x_{i}\\}^{d}_{i=1}$ and the power mask matrix $\mathbf{PM}$ with
exponantiating selected features by $\\{q_{i}\\}^{d}_{i=1}$. A resulting
monomial matrix $\mathbf{MM}$ is made by element-wise exponentiation
$\mathbf{MM}=\mathbf{TM}^{\mathbf{PM}}$. Thus, when the resulting matrix is
squeezed by elements-product across the $2^{nd}$ dimension, we get the
monomials of the input features $\psi_{m}(\mathbf{x})$. Using the randomly
initialized vector of learnable coefficients $\mathbf{a}_{m}$ we apply the
Hadamard product as in (5).

Thus, with $d$ features as an input, the LEFM layer gives $D$ output features
with learnable coefficients $\\{a_{i}\\}^{D}_{i=1}$. As illustrated in Fig. 1,
the expanded output is passed to the arbitrary deep NN to jointly train its
weights and LEFM layer’s coefficients.

Figure 2: Fleiss kappa statistics for evaluating inter-annotators agreement.
Left and right plots represent statistics per image in CryoNuSeg and CoCaHis
datasets, respectively. The Fleiss kappa for the entire CryoNuSeg is 0.8012
and for CoCaHis 0.7405. Former indicates an almost perfect inter-annotator
agreement, whereas the latter indicates a substantial inter-annotator
agreement.

###  4.2 Deep neural networks

Within the scope of this paper, the LEFM layer is combined with several types
of slightly modified neural networks. In DeepLabv3+ [20], a deep network has
the spatial pyramid pooling module combined with the encoder-decoder
structure. Feature extraction at arbitrary resolution was enabled by atrous
convolution layers contained in the encoder module [20]. On the other hand, a
simple decoder recovers detailed boundaries of an object. A popular UNet model
[17] employs a symmetric U-shaped encoder-decoder for precise feature
localization. Skip connections help in preserving context at the same level of
the model’s depth [17]. Natural extension of the UNet is a nested model.
Introduced in [18], UNet++ model proposes additional nodes and skip
connections to the original UNet. That facilitates predicting the output from
different levels of depth. Training the entire network by combining all
outputs, the UNet++ model outperforms UNet with a cost of a larger number of
parameters. Finally, MA-net [21] incorporates a self-attention mechanism.
Based on the attention mechanism, it collects rich contextual dependencies
while adaptively integrating local features. Model implements Position-wise
Attention Block and Multi-scale Fusion Attention Block. Prior is used to model
the feature interdependencies spatially, and the former to capture channel
dependencies using semantic feature fusion [21].

All mentioned models can implement different backbones for feature extraction
to their core. When comparing a model with and without the LEFM layer, it is
important to keep the backbone fixed. MobileNetV2 [36] was implemented in
DeepLabv3+ model, whereas other models have DenseNet201 [37] backbone. All
backbone’s weights are pretrained on Imagenet classification problem [38] and
fine-tuned with the rest of the model. Thus, the total number of trainable
parameters in DeepLabv3+ model is ~4.4M, in UNet ~28.5M, in UNet++ ~48.5M, and
in MA-net ~133.7M. Furthermore, LEFM-Nets, with mentioned architectures and
$m=3$, contribute to an increase of the number of learnable parameters in the
amount of: 0.11%, 0.18%, 0.11%, and 0.04% relative to the original networks.
In the application considered herein, it is modest when compared to the
reported total number of trainable parameters of original models.

###  4.3 Datasets

This paper is focused on segmentation of color images of H&E stained frozen
sections. Thereby, we used the only two publicly available datasets: CoCaHis
[16, 39] and CryoNuSeg [40]. Datasets are divided into the train and test
sets. Furthermore, during the training process, 20% of the train set is left
for validation and early stopping purposes.

####  4.3.1 CoCaHis

Dataset comprises 82 images with dimensions $1037\times 1388$. Training set
has 58 images belonging to 13 patients. The rest of 24 images from 6 patients
are contained in the test set. Sets are disjointed for both images and
patients. Seven annotators generated the pixel-wise labels for each image. One
and zero indicate cancerous and non-cancerous pixels, respectively. Resulting
average Fleiss kappa statistic [41], presented in Section 4.7 and Fig. 2,
supports usage of majority voting due to the substantial inter-annotator
agreement [42].

####  4.3.2 CryoNuSeg

Collected from The Cancer Genome Atlas111https://www.cancer.gov/tcga, the
dataset contains 30 whole-slide frozen section images. Afterwards, they are
cropped to the most representative window. Hence, the dataset comprises 30
images of the size $515\times 512$ pixels that correspond to 10 organs. The
problem formulated by this dataset is segmentation of nuclei in different
images. Therefore, the ground truth generation is similar to CoCaHis. The
Fleiss kappa statistic, presented in Section 4.7 and Fig. 2, suggests an
almost perfect agreement between three annotations, and that is why the
majority vote is used. To ensure the representation of every organ in both
train and test set, one image per each organ is left out for the test set (10
images), whereas the remaining images are included in the training set (20
images).

###  4.4 Statistics and Metrics

####  4.4.1 Statistical significance analysis

To determine if inclusion of the LEFM layer yields statistically better
performance when compared to the original models, we used a one-way analysis
of variance (ANOVA) test. All models were trained 10 times with different
initial weights and different subsets for validation. Null hypothesis was that
two groups of results, obtained by the LEFM-Net and the original deep network,
have equal means. For all tests, a level of significance $\alpha=0.05$ was
used.

####  4.4.2 Evaluation metrics

All metrics used for evaluation were calculated using standard $TP$, $TN$,
$FN$, and $FP$ notation. $TP$ stands for a number of correctly diagnosed
cancerous pixels. $TN$ indicates a number of non-cancerous pixels correctly
diagnosed as non-cancerous. $FN$ stands for a number of cancerous pixels
incorrectly diagnosed as non-cancerous. Finally, $FP$ indicates the number of
noncancerous pixels incorrectly diagnosed as cancerous. Thus, the following
micro measures were used for models’ performance validation:

  * •

a harmonic mean of precision and sensitivity (recall) presented as a $F_{1}$
score:

$F_{1}=\frac{2\cdot TP}{2\cdot TP+FP+FN},$ (8)

  * •

balanced accuracy (BACC) as an arithmetic mean of sensitivity (SE) and
specificity (SP):

$SE=\frac{TP}{TP+FN},\ SP=\frac{TN}{{TN}+{FP}},$

$BACC=\frac{SE+SP}{2},$ (9)

  * •

and Precision score, which measures the classifier’s capacity not to
categorize a sample as positive if it is negative:

$PREC=\frac{TP}{TP+FP}.$ (10)

For all the metrics, 0 indicates the worst performance and 1 indicates the
best performance.

Figure 3: Target image for stain normalization chosen by pathologist.
Selection is based on the quality-of-staining criterion.

###  4.5 Data preprocessing

As an important part of the computational pathology pipeline, stain
normalization was performed to eliminate the variability of staining and
experimental variations. The structure preserved color normalization method
[43] was used on both CoCaHis and CryoNuSeg datasets. A target image, shown in
Fig. 3, was carefully chosen by an experienced pathologist, ensuring a well
stained specimen for the mentioned algorithm. Fig. 4 shows original and stain-
normalized images from the mentioned datasets. Stain-transfer from the target
image helps models during the training to cope with the color variation
problem.

Prior to the training, every image was scaled to the range [0,1].
Additionally, random shifting, scaling, rotating and flipping (horizontal,
vertical) were applied as part of the data augmentation procedure. Chosen
parameters are presented in Section 4.6.

(a)

(b)

(c)

(d)

Figure 4: Visualization of raw and stain normalized images. Subfigures (a) and
(c) show original images from CoCaHis and CryoNuSeg, respectively. Their
stain-normalized equivalents are shown in subfigures (b) and (d). Table 1:
Performance scores of four different models on CryoNuSeg and CoCaHis datasets.
Each model had 10 runs with different initial weights. Backbones were
initialized on the pretrained ImageNet classification problem. Variable
$m\in\\{2,3\\}$ indicates level of expansion. The best result for each metric
and each dataset is in bold font.

|  | Original network | LEFM-Net m=2 | LEFM-Net m=3
---|---|---|---|---
| Metric | CryoNuSeg | CoCaHis | CryoNuSeg | CoCaHis | CryoNuSeg | CoCaHis
| BACC | 87.24 $\pm$ 0.54 | 86.06 $\pm$ 1.60 | 86.99 $\pm$ 0.64 | 87.24 $\pm$ 1.55 | 87.52 $\pm$ 0.74 | 88.19 $\pm$ 0.70
| F1 | 82.34 $\pm$ 0.60 | 80.75 $\pm$ 1.44 | 82.11 $\pm$ 0.67 | 81.89 $\pm$ 1.69 | 82.60 $\pm$ 0.69 | 82.85 $\pm$ 1.38
DeepLabv3+ | PREC | 85.49 $\pm$ 0.39 | 84.48 $\pm$ 4.70 | 85.74 $\pm$ 0.60 | 83.29 $\pm$ 2.96 | 85.29 $\pm$ 0.79 | 82.76 $\pm$ 3.96
|  BACC | 88.99 $\pm$ 0.87 | 86.55 $\pm$ 1.46 | 89.05 $\pm$ 0.95 | 85.58 $\pm$ 2.32 | 88.73 $\pm$ 0.95 | 88.67 $\pm$ 1.64
| F1 | 84.95 $\pm$ 0.82 | 79.94 $\pm$ 1.70 | 85.02 $\pm$ 0.87 | 78.53 $\pm$ 3.72 | 84.79 $\pm$ 0.95 | 82.59 $\pm$ 1.67
UNet | PREC | 88.12 $\pm$ 0.74 | 78.60 $\pm$ 5.71 | 88.17 $\pm$ 1.37 | 77.35 $\pm$ 9.59 | 88.73 $\pm$ 1.28 | 80.08 $\pm$ 4.19
| BACC | 88.98 $\pm$ 0.57 | 82.01 $\pm$ 6.57 | 89.29 $\pm$ 0.60 | 86.99 $\pm$ 4.66 | 88.88 $\pm$ 0.56 | 88.23 $\pm$ 2.13
| F1 | 84.94 $\pm$ 0.52 | 74.04 $\pm$ 9.71 | 85.27 $\pm$ 0.61 | 79.94 $\pm$ 7.37 | 84.93 $\pm$ 0.61 | 81.55 $\pm$ 4.38
UNet++ | PREC | 88.09 $\pm$ 0.72 | 75.64 $\pm$ 17.35 | 88.04 $\pm$ 0.57 | 76.69 $\pm$ 11.94 | 88.48 $\pm$ 1.08 | 78.23 $\pm$ 10.51
| BACC | 89.30 $\pm$ 0.44 | 88.02 $\pm$ 1.22 | 89.31 $\pm$ 0.38 | 88.69 $\pm$ 1.63 | 89.41 $\pm$ 0.29 | 89.36 $\pm$ 1.27
| F1 | 85.12 $\pm$ 0.51 | 82.75 $\pm$ 1.10 | 85.27 $\pm$ 0.39 | 84.33 $\pm$ 1.98 | 85.35 $\pm$ 0.25 | 84.96 $\pm$ 1.14
MA-net | PREC | 87.36 $\pm$ 0.58 | 83.16 $\pm$ 2.44 | 87.93 $\pm$ 0.46 | 86.52 $\pm$ 1.60 | 87.76 $\pm$ 0.53 | 86.15 $\pm$ 2.00

###  4.6 Implementation with training and test details

LEFM layer is implemented in Python using the Pytorch framework [44]. All
existing deep networks are inherited and modified from segmentation-models-
pytorch library [45]. For faster convergence [16], weights of backbones are
initialized on ImageNet classification problem [38] and then fine-tuned with
the rest of the LEFM-Net.

Before the training, images were preprocessed as described in Section 4.5. For
the evaluation of the statistical significance, all models were trained with
10 different random seeds. Thus, the initialization of newly added weights,
coefficients, and choice of patches withheld for validation set were different
in every run. In Table 1, we show the average performance of models on
selected metrics (see Section 4.4.2). For each dataset, the corresponding test
sets are the same for every model.

We selected Adam [46] optimizer for the training procedure. To equally punish
the underperformance in terms of false positives and false negatives, we used
Dice loss function [47, 48]. The hyperparameters for every model were set as
follows:

  * •

maximum number of epochs equals to $3\cdot 10^{4}$

  * •

starting learning rate was set to $10^{-3}$

  * •

learning rate is reduced when the loss plateau is reached

    * –

waiting patience for improvement is set to 20 epochs

    * –

reduction factor is set to 0.5

    * –

minimum learning rate equals $10^{-6}$

  * •

factor for $L_{2}$ weight decay is set to $10^{-4}$

  * •

Early stopping after 40 epochs when there is no improvement on the validation
set

  * •

data augmentation

    * –

shift limit = 0.2

    * –

scale limit = 0.2

    * –

rotation limit = 30∘

    * –

horizontal and vertical flip

    * –

probability = 0.5

  * •

LEFM order $m\in\\{2,3\\}$

###  4.7 Results

Results of exhaustive training and testing of four different models, with and
without the LEFM layer, on two datasets and 10 random seeds are shown in Table
1. Models are ordered by the number of parameters. As mentioned in Section
4.2, DeepLabv3+ has the least and MA-net the most trainable weights. When
comparing these two models, a noticeable increase in scores can always be
found in the model with larger capacity, i.e. MA-net.

It can be seen that four different versions of LEFM-Nets, compared to the
original networks, yield better scores on two datasets. Higher order features
are important for better end results. As it can be seen in Table 1, regardless
of the level of expansion, LEFM-Net outperforms original deep models. Also,
nearly half of the observed metrics for CryoNuSeg dataset and almost all of
them for CoCaHis dataset are better when the level of expansion is $m=3$.

Tests performed on CryoNuSeg show that all LEFM-Nets are on average better
than the original network. ANOVA statistical testing indicates that, with
statistical significance $\alpha=0.05$, all LEFM-Nets significantly outperform
original models on CoCaHis dataset for $m=3$. Tests are performed on $F_{1}$
and BACC metrics. Hence, introduction of the LEFM layer seems to be justified
for intraoperative pixel-based diagnostics with deep learning models.

The training time depends on the number of training parameters in the model,
dataset, batch- and patch size, and on the degree of expansion $m$. With
NVIDIA Tesla V100S GPU, training time for one run ranges from 2 hours, when
training DeepLabv3+, to 20 hours when training MA-net.

##  5 Discussion and Conclusion

Figure 5: Visualization of coefficients learned during the training on
CryoNuSeg dataset. DeepLabv3+, UNet, UNet++, and MA-net were trained with
expansion layer of degree $m=3$. During the training, $L_{2}$ norm was used
for weight decay. Red ($R$), green ($G$) and blue ($B$) stand for $x_{1}$,
$x_{2}$ and $x_{3}$ features, respectively.

The core idea behind the LEFM layer is generation of more explicit features in
a data-driven manner, i.e. letting the network to decide on their importance
for the image segmentation problem at hand. Towards that, we illustrate in
Fig. 5 the relative importance of explicit features induced by the LEFM or
order $m=3$ on the CryNuSeg dataset. For the sake of interpretability, we name
the abstract features in the input space $\\{x_{1},x_{2},x_{3}\\}$ by the
colors they represent {red, green, blue}. To interpret Fig. 5 it is important
to notice that in images of H&E stained specimens, nucleus, cytoplasm and
glandular structures appear respectively blue-purple, pink and white.
Intensely red structures may also represent red blood cells [49]. Since red
(R) and blue (B) colors are dominant in the purple color formation, they are
dominantly expressed in Fig. 5. However, justification for using the LEFM
layer is because some explicit features, that do not exist in the input data
space (monomials of the original features), are also important when the model
is segmenting the image. As seen in Fig. 5, the networks are using them up to
certain extents. Hence, when the new features were added, i.e. number of
channels is increased, LEFM-Net is expected to be guided towards better
predictions while training. Thus, the level of expansion $m$ is a
hyperparameter for cross-validation. However, there is a memory limitation to
what degree can features be expanded (see Eq. (4)). Thus, we limited our
experiments to the $m\in\\{2,3\\}$.

Interpretability of features in the domain of histopathological pixel-wise
diagnostics is straightforward. Newly added features are nonlinear mixtures of
the original features from the input space. Therefore, they can be interpreted
as mixtures of colors needed for discriminating between cancer and non-cancer
pixels. Using the previously explained coloring for nuclei, cytoplasm and
glandular structures, it is clear that blue-purple, pink and white colors
cannot be explained in terms of $R$, $G$ and $B$ features only. Thus, networks
also used $RG$, $RB$ and $GB$ features, as seen in Fig. 5. Since those
features do not exist in the input data space, the LEFM layer helps the
original network by providing the new medically meaningful features.

It is important to mention that feature expansion brings slightly increased
computational complexity as there are more channels to the input of existing
neural networks. Therefore, besides newly introduced coefficients in the LEFM
layer, new neurons are added in every existing network to adapt for the
expanded input. Thus, specific number of new learnable weights depends on the
network’s architecture. However, as discussed in Section IV.B the increase is
modest.

To conclude, this work introduced a data-adaptive and learnable explicit
feature map layer. Besides the features from the input data space, it
generates higher order features and helps the existing network to improve
segmentation/classification performance. New features are predefined by order
of expansion and their importance is learned through the training process.
Learned coefficients identify whether the feature is important for
classification. Overall, if the original features are interpretable, then
their monomials should be interpretable as well. The same applies to color
mixtures of H&E stains in the domain of histopathological image segmentation.
By performing exhaustive validation on two different datasets and across
multiple runs, introducing the LEFM layer to existing deep models seems to be
justified. Presented statistical significance analysis further supports this
statement.

## References

  * [1] D. Shen, G. Wu, and H.-I. Suk, “Deep learning in medical image analysis,” _Annual review of biomedical engineering_ , vol. 19, pp. 221–248, 2017.
  * [2] G. Litjens, C. I. Sánchez, N. Timofeeva, M. Hermsen, I. Nagtegaal, I. Kovacs, C. Hulsbergen-Van De Kaa, P. Bult, B. Van Ginneken, and J. Van Der Laak, “Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis,” _Scientific reports_ , vol. 6, no. 1, pp. 1–11, 2016.
  * [3] J. De Fauw, J. R. Ledsam, B. Romera-Paredes, S. Nikolov, N. Tomasev, S. Blackwell, H. Askham, X. Glorot, B. O’Donoghue, D. Visentin _et al._ , “Clinically applicable deep learning for diagnosis and referral in retinal disease,” _Nature medicine_ , vol. 24, no. 9, pp. 1342–1350, 2018.
  * [4] S. Mishra, Y. Zhang, D. Z. Chen, and X. S. Hu, “Data-driven deep supervision for medical image segmentation,” _IEEE Transactions on Medical Imaging_ , 2022.
  * [5] R. Gu, G. Wang, T. Song, R. Huang, M. Aertsen, J. Deprest, S. Ourselin, T. Vercauteren, and S. Zhang, “Ca-net: Comprehensive attention convolutional neural networks for explainable medical image segmentation,” _IEEE transactions on medical imaging_ , vol. 40, no. 2, pp. 699–711, 2021.
  * [6] G. N. Gunesli, C. Sokmensuer, and C. Gunduz-Demir, “Attentionboost: Learning what to attend for gland segmentation in histopathological images by boosting fully convolutional networks,” _IEEE Transactions on Medical Imaging_ , vol. 39, no. 12, pp. 4262–4273, 2020.
  * [7] W. Li, J. Li, Z. Wang, J. Polson, A. E. Sisk, D. P. Sajed, W. Speier, and C. W. Arnold, “Pathal: An active learning framework for histopathology image analysis,” _IEEE Transactions on Medical Imaging_ , 2021.
  * [8] H. Yang, J.-Y. Kim, H. Kim, and S. P. Adhikari, “Guided soft attention network for classification of breast cancer histopathology images,” _IEEE transactions on medical imaging_ , vol. 39, no. 5, pp. 1306–1315, 2020.
  * [9] C. Zhu, W. Chen, T. Peng, Y. Wang, and M. Jin, “Hard sample aware noise robust learning for histopathology image classification,” _IEEE Transactions on Medical Imaging_ , 2021.
  * [10] X. Guo, C. Yang, Y. Liu, and Y. Yuan, “Learn to threshold: Thresholdnet with confidence-guided manifold mixup for polyp segmentation,” _IEEE Transactions on Medical Imaging_ , vol. 40, no. 4, pp. 1134–1146, 2021.
  * [11] C. Rudin, “Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead,” _Nature Machine Intelligence_ , vol. 1, no. 5, pp. 206–215, 2019.
  * [12] E. Tjoa and C. Guan, “A survey on explainable artificial intelligence (xai): Toward medical xai,” _IEEE transactions on neural networks and learning systems_ , vol. 32, no. 11, pp. 4793–4813, 2020.
  * [13] M. Veta, J. P. Pluim, P. J. Van Diest, and M. A. Viergever, “Breast cancer histopathology image analysis: A review,” _IEEE transactions on biomedical engineering_ , vol. 61, no. 5, pp. 1400–1411, 2014.
  * [14] J. X. Qiu, H.-J. Yoon, P. A. Fearn, and G. D. Tourassi, “Deep learning for automated extraction of primary sites from cancer pathology reports,” _IEEE journal of biomedical and health informatics_ , vol. 22, no. 1, pp. 244–251, 2018.
  * [15] D. Komura and S. Ishikawa, “Machine learning methods for histopathological image analysis,” _Computational and structural biotechnology journal_ , vol. 16, pp. 34–42, 2018.
  * [16] D. Sitnik, G. Aralica, M. Hadžija, M. P. Hadžija, A. Pačić, M. M. Periša, L. Manojlović, K. Krstanac, A. Plavetić, and I. Kopriva, “A dataset and a methodology for intraoperative computer-aided diagnosis of a metastatic colon cancer in a liver,” _Biomedical Signal Processing and Control_ , vol. 66, p. 102402, 2021.
  * [17] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in _International Conference on Medical image computing and computer-assisted intervention_. Springer, 2015, pp. 234–241.
  * [18] Z. Zhou, M. M. Rahman Siddiquee, N. Tajbakhsh, and J. Liang, “Unet++: A nested u-net architecture for medical image segmentation,” in _Deep learning in medical image analysis and multimodal learning for clinical decision support_. Springer International Publishing, 2018, pp. 3–11.
  * [19] Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and J. Liang, “Unet++: Redesigning skip connections to exploit multiscale features in image segmentation,” _IEEE transactions on medical imaging_ , vol. 39, no. 6, pp. 1856–1867, 2020\.
  * [20] L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder-decoder with atrous separable convolution for semantic image segmentation,” in _Proceedings of the European conference on computer vision (ECCV)_ , 2018, pp. 801–818.
  * [21] T. Fan, G. Wang, Y. Li, and H. Wang, “Ma-net: A multi-scale attention network for liver and tumor segmentation,” _IEEE Access_ , vol. 8, pp. 179 656–179 665, 2020.
  * [22] T. Wang, L. Zhang, and W. Hu, “Bridging deep and multiple kernel learning: A review,” _Information Fusion_ , vol. 67, pp. 3–13, 2021.
  * [23] B. Pan, J. Lai, and P. C. Yuen, “Learning low-rank mercer kernels with fast-decaying spectrum,” _Neurocomputing_ , vol. 74, no. 17, pp. 3028–3035, 2011.
  * [24] B. Kulis, M. A. Sustik, and I. S. Dhillon, “Low-rank kernel learning with bregman matrix divergences.” _Journal of Machine Learning Research_ , vol. 10, no. 2, 2009.
  * [25] Z. Li, U. Kruger, L. Xie, A. Almansoori, and H. Su, “Adaptive kpca modeling of nonlinear systems,” _IEEE Transactions on Signal Processing_ , vol. 63, no. 9, pp. 2364–2376, 2015.
  * [26] F. R. Bach, G. R. Lanckriet, and M. I. Jordan, “Multiple kernel learning, conic duality, and the smo algorithm,” in _Proceedings of the twenty-first international conference on Machine learning_ , 2004, p. 6.
  * [27] I. Rebai, Y. BenAyed, and W. Mahdi, “Deep multilayer multiple kernel learning,” _Neural Computing and Applications_ , vol. 27, no. 8, pp. 2305–2314, 2016.
  * [28] A. G. Wilson, Z. Hu, R. Salakhutdinov, and E. P. Xing, “Deep kernel learning,” in _Artificial intelligence and statistics_. PMLR, 2016, pp. 370–378.
  * [29] B. Bohn, C. Rieger, and M. Griebel, “A representer theorem for deep kernel learning,” _The Journal of Machine Learning Research_ , vol. 20, no. 1, pp. 2302–2333, 2019.
  * [30] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” _Proceedings of the IEEE_ , vol. 86, no. 11, pp. 2278–2324, 1998.
  * [31] B. Scholkopf and A. J. Smola, _Learning with kernels: support vector machines, regularization, optimization, and beyond_. MIT press, 2018.
  * [32] G. Kimeldorf and G. Wahba, “Some results on tchebycheffian spline functions,” _Journal of mathematical analysis and applications_ , vol. 33, no. 1, pp. 82–95, 1971.
  * [33] A. Vedaldi and A. Zisserman, “Efficient additive kernels via explicit feature maps,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 34, no. 3, pp. 480–492, 2012.
  * [34] T. M. Cover, “Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition,” _IEEE transactions on electronic computers_ , no. 3, pp. 326–334, 1965.
  * [35] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in _Proceedings of the 32nd International Conference on Machine Learning_ , ser. Proceedings of Machine Learning Research, F. Bach and D. Blei, Eds., vol. 37. Lille, France: PMLR, 07-09 Jul 2015, pp. 448–456.
  * [36] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 4510–4520.
  * [37] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 4700–4708.
  * [38] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in _2009 IEEE Conference on Computer Vision and Pattern Recognition_ , 2009, pp. 248–255.
  * [39] D. Sitnik, G. Aralica, M. Hadžija, M. P. Hadžija, A. Pačić, M. M. Periša, L. Manojlović, K. Krstanac, A. Plavetić, and I. Kopriva, “Cocahis dataset,” https://cocahis.irb.hr, 2021, accessed: 2022-04-12.
  * [40] A. Mahbod, G. Schaefer, B. Bancher, C. Löw, G. Dorffner, R. Ecker, and I. Ellinger, “Cryonuseg: A dataset for nuclei instance segmentation of cryosectioned h&e-stained histological images,” _Computers in Biology and Medicine_ , vol. 132, p. 104349, 2021.
  * [41] J. L. Fleiss, “Measuring nominal scale agreement among many raters.” _Psychological bulletin_ , vol. 76, no. 5, p. 378, 1971.
  * [42] J. R. Landis and G. G. Koch, “The measurement of observer agreement for categorical data,” _biometrics_ , pp. 159–174, 1977.
  * [43] A. Vahadane, T. Peng, A. Sethi, S. Albarqouni, L. Wang, M. Baust, K. Steiger, A. M. Schlitter, I. Esposito, and N. Navab, “Structure-preserving color normalization and sparse stain separation for histological images,” _IEEE Transactions on Medical Imaging_ , vol. 35, no. 8, pp. 1962–1971, 2016\.
  * [44] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-performance deep learning library,” in _Advances in Neural Information Processing Systems 32_. Curran Associates, Inc., 2019, pp. 8024–8035.
  * [45] P. Yakubovskiy, “Segmentation models pytorch,” https://github.com/github/qubvel/segmentation_models.pytorch, 2020.
  * [46] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 2014.
  * [47] H. Seo, M. Bassenne, and L. Xing, “Closing the gap between deep neural network modeling and biomedical decision-making metrics in segmentation via adaptive loss functions,” _IEEE Transactions on Medical Imaging_ , vol. 40, no. 2, pp. 585–593, 2021.
  * [48] T. Eelbode, J. Bertels, M. Berman, D. Vandermeulen, F. Maes, R. Bisschops, and M. B. Blaschko, “Optimization for medical image segmentation: Theory and practice when evaluating with dice score or jaccard index,” _IEEE Transactions on Medical Imaging_ , vol. 39, no. 11, pp. 3679–3690, 2020.
  * [49] S. Kothari, J. H. Phan, T. H. Stokes, A. O. Osunkoya, A. N. Young, and M. D. Wang, “Removing batch effects from histopathological images for enhanced cancer diagnosis,” _IEEE Journal of Biomedical and Health Informatics_ , vol. 18, no. 3, pp. 765–772, 2014.