#  A State-of-the-art Survey of Artificial Neural Networks for Whole-slide
Image Analysis: from Popular Convolutional Neural Networks to Potential Visual
Transformers

Xintong Li1, Weiming Hu1, Chen Li1, Tao Jiang2, Hongzan Sun3, Xiaoyan Li3,
Xinyu Huang4, Marcin Grzegorzek4
1\. Microscopic Image and Medical Image Analysis Group, College of Medicine
and Biological Information Engineering, Northeastern University, Shenyang,
China
2\. Chengdu University of Information Technology, Chengdu, CHina 3\. China
Medical University, Shenyang, China
4\. Institute for Medical Informatics, University of Luebeck, Luebeck, Germany
Corresponding author: Chen Li, E-mail<EMAIL_ADDRESS>
###### Abstract

In recent years, with the advancement of computer-aided diagnosis (CAD)
technology and whole slide image (WSI), histopathological WSI has gradually
played a crucial aspect in the diagnosis and analysis of diseases. To increase
the objectivity and accuracy of pathologists’ work, artificial neural network
(ANN) methods have been generally needed in the segmentation, classification,
and detection of histopathological WSI. In this paper, WSI analysis methods
based on ANN are reviewed. Firstly, the development status of WSI and ANN
methods is introduced. Secondly, we summarize the common ANN methods. Next, we
discuss publicly available WSI datasets and evaluation metrics. These ANN
architectures for WSI processing are divided into classical neural networks
and deep neural networks (DNNs) and then analyzed. Finally, the application
prospect of the analytical method in this field is discussed. The important
potential method is Visual Transformers.

###### keywords:

Whole-slide image analysis , computer-aided diagnosis , image segmentation ,
image classification , object detection

††journal: Elsevier Journal

##  1 Introduction

###  1.1 Brief Knowledge

In pathology, biopsy diagnosis is the gold standard for cancer diagnosis [73].
Conventionally, the inspection of slides under a microscope has been a typical
method in histopathology [28, 38], and over the past decade, automated
analysis of histopathology has become one of the fastest-growing areas in
medical image calculating. _Computer-aided Diagnosis_ (CAD) can reduce the
workload of pathologists and the rate of misdiagnosis [79]. The digital
version to slide is called _Whole-slide Image_ (WSI). It allows pathology
slides to be stored digitally on a computer.

WSI, also known as virtual microscopy, involves digitizing or scanning slides
to make digital slides for human observation and automatic image analysis.
Currently, WSI technology has advanced to the point where digital slide
scanners can make high-resolution digital images in a relatively short time.
The digital slide show is designed to simulate a light microscope. These WSIs
can be checked on the computer screen with the assistance of persons working
in the relevant field. The viewer can examine the whole slide in a manner
comparable to that of a light microscope and observe it in any direction and
at different magnification [47, 101, 100]. However, due to the large
resolution level and pixel size of WSI, this poses a huge challenge for image-
level processing. If the thumbnails are down-sampled, a lot of information
will be lost. Therefore, the usual processing method is to intercept WSI into
patches with fine details for analysis [130]. The process of this interception
is shown in the Figure. 1.

Figure 1: The process of cutting WSI into a patch. The figure matches to Fig.1
in [68].

In recent years, WSI has provided convenience in various fields, especially in
the field of pathological research. For example, many people can open the same
WSI at different places at the same time. This feature helps pathologists
remote viewing, conferences, and E-learning. And its high quality, low storage
cost, non-destructive, and fading characteristics are conducive to
preservation. In the part of patients, the nursing of patients is further
improved. However, the regulatory issues and practice standards of WSI are
also issues that need attention and consideration [95].

The use of CAD for WSI viewing is an efficient, accurate, and intuitive
method. It can be used to diagnose different diseases, such as lung cancer,
gastric cancer, colon cancer, and breast cancer. For example, in the work of
[131], a fasting and efficient classification method for WSI lung cancer
images is proposed. Firstly, the patch-based _full convolutional network_
(FCN) is used to retrieve images at the patch-level, which is used to extract
depth features and generate WSI descriptors. Then the descriptor is input into
the Random Forest (RF) [25] classifier used for image-level prediction to
classify tumors. Finally, the high accuracy rate of this method is 97.3%. The
study of [105], proposed a _Convolutional Neural Network_ (CNN) [14]
architecture for WSI classification and necrosis detection of gastric cancer
histopathology. This method is compared with the use of traditional features
combined with RF for classification and AlexNet [71] for comparative analysis.
According to the final experiment, the proposed CNN architecture has obtained
favorable results. The overall classification accuracy of cancer
classification is $69.90\%$, and the overall classification accuracy of
necrosis detection is $81.44\%$. In the domain of colon cancer, the aim of
[64] propose the classification method derived from Support Vector Machine
(SVM) [18] to classify colon cancer and non-colon cancer WSI. Then 18 simple
features such as gray-level mean, gray-level variance, etc., and 16 texture
features are selected. These 16 features are extracted by the Gray-level Co-
occurrence Matrix (GLCM). Then the extracted features are put together as
feature sets. The final experimental results are that the mean values of
accuracy, recall, and F-score are $96.67\%$, $83.33\%$, and $89.51\%$,
respectively. The experimental results show the important contribution of this
method in classifying colon cancer WSI. The author of [9] proposed a multi-
scale superpixel classification method to detect the epithelial area of Ductal
Carcinoma In Situ (DCIS) WSI, then classify it and detect the scope of DCIS.
The final test results detected abnormal WSI in $80.00\%$ and $83.00\%$ of
DCIS lesions, verifying the effectiveness of the method.

###  1.2 The Development of WSI Analysis Using ANN

Based on the rapid development of these CAD methods, more and more scholars
begin to enter the field of _Machine Learning_ (ML). ML is a special research
of how computers simulate or realize human learning behavior, to achieve
modern knowledge or skills, reconstruct the current knowledge framework, and
constantly increase their achievement. ML can separate into supervised
learning, unsupervised learning, and reinforcement learning according to the
types of data processed [87, 13]. As shown in Figure. 2.

Figure 2: The structure of ANN technology in the ML system.

Among different machine learning methods, _Artificial Neural Network_ (ANN)
[84] is an important technology to simulate the intelligent principle of the
human brain. The meaning of ANNs is fundamentally proposed from the biological
field. In this field, neural networks play a critical part in the human body.
The human body does what it should do to assist neural networks. A neural
network is a network that millions of linked neurons working together. With
the help of these neurons, full coordinate processing is done in the human
body, which is the best example of parallel processing [106]. Similar to
biological neurons, ANNs have artificial neurons that also receive inputs from
other components or other artificial neurons, and then convert the results
into outputs via transfer functions after the inputs are weighted and added
[30]. The most basic neural network model is shown in Figure. 3, which usually
consists of three parts: The input layer, the hidden layer, and the output
layer. When ANN is operated on the computer, it is not used for specific
functions to program. Instead of this function, they are trained as a whole in
the dataset until they understand the data pattern presented to them. After
training them, new patterns can be presented to them for functions such as
classification, segmentation, and detection [65].

Figure 3: The most basic neural network model.

Currently, ANNs are used to assist pathologists to browse the development
trend of WSI as shown in Figure. 4. Since 2006, ANNs have been used to assist
pathologists in WSI. The three most commonly used fields are image
classification, segmentation, and detection. From 2006 till now, the number of
cases in these three main applications has increased year by year. Among them,
the cases applied to classification are the most, followed by detection, then
segmentation, and other applications such as machine-generated text [82] and
automatic labeling [107]. The growth rate of its number is accelerating year
by year, which also shows the progress of technology.

Figure 4: The pathologist is assisted by ANN to view the development trend of
WSI.

###  1.3 Motivation of This Review

As far as we know, there are some literature reviews on the related studies of
ANNs that assist pathologists to view the work of WSI (e.g., the reviews in
[141, 1, 48, 39]). In the following part, we go through the survey papers.

The survey of [141] reviews the methods of breast cancer histopathological
image analysis based on ANNs, including segmentation, classification, and
dataset partition. More than 130 papers are summarized, but they are only
about breast cancer, not about other kinds of cancer.

The survey of [1] summarizes the important algorithms of CAD of lung tissue,
including traditional deep learning algorithms and ANNs. More than 20 papers
are summarized, but in this paper, we only focus on the tasks based on ANNs
and WSI technology.

In [48], the comparison of pathological diagnosis results by WSI and light
microscopy are reviewed to measure the consistency of the two methods. More
than 30 studies have been summarized, but the deep learning algorithms of CAD
are not analyzed.

The survey of [39] summarizes the application of WSI technology platforms.
According to clinical and non-clinical classification. More than 60 papers are
summarized, but they are all about the hardware of WSI systems and the
advantages, limitations, and prospects of their development.

From the current review papers, we can find that many researchers are
concerned about the device hardware, development status, and trend of WSI
technology. In these review papers, many related works are methodically
summarized and discussed. However, all of these research papers are not
intended to introduce the use of ANN computer-aided pathologists to review WSI
technology in detail. Therefore, we propose this review paper, combined with
the related work done by ANN and WSI in the past few decades, using CAD to
analyze. This survey summarizes more than 50 related works from 2006 to 2020.
The process of selecting papers is shown in Figure. 5. The audience of this
review is related researchers and medical professionals in the field of
medical imaging.

Figure 5: The systematic flow chart of paper selection for our work.

###  1.4 Structure of This Review

This structure of the paper is as follows: Sect. 2 illustrates the common ANNs
methods. Sect. 3 summarizes the related datasets and commonly used evaluation
methods. Sect. 4, and 5 present related work of ANN using WSI and CAD
technology. After the overview of different works, the most frequently used
approaches are analyzed in Sect. 6. Finally, Sect. 7 and 8 conclude this
review with prospective future direction.

##  2 Common ANNs methods

In the previous description, we found that the CNN model is the most commonly
used supervised machine learning model for WSI of CAD. Its inspiration comes
from the tissue of the animal visual cortex [59, 41], it is designed to
automatically and adaptively learn the spatial levels of features from low-
level to high-level patterns. In 1980, a new neuron was considered to be the
predecessor of CNN in [42]. LeNet was the pioneering work of LeCun et al. in
CNNs in 1990 [75], which was later improved. So far, CNN has been more widely
used.

CNN, as the name implies. It uses a mathematical linear operation called
convolution. This kind of linear operation can replace the original matrix
multiplication operation in at least one layer of CNN [31]. It generally
includes a convolutional layer, pooling layer, and fully connected layer [93].
The convolutional layer is the most important one. The convolution layer is
mainly used to extract the input data. Here, the input data contains the
features of multiple convolution cores. Each element of the convolution kernel
corresponds to its weight co-efficient and deviation. It’s like neurons in a
feedforward neural network. Each neuron in the area of the current convolution
layer near the previous convolution layer is connected to several neurons. The
size of the convolution layer determines the size of this area, namely the
receptive field, which can be compared with the receptive field of visual
cortex cells [50]. When the convolution kernel works, it will scan the input
features in a fixed time. The operation also includes multiplication and
summation of the matrix elements generated by the input features in the
receiving domain and then stacking the deviations [49].

There are two kinds of pooling layers, which are maximum pooling and average
pooling. Its main function is to reduce the dimension, and reduce the
dimension by downsampling the convolutional results [31]. The full
convolutional layer is to expand the output of the convolutional layer and
pooling layer into a one-dimensional form, followed by a regression network or
classification network with the same structure as the ordinary network, which
is generally connected behind the pooling layer [136].

LeNet is the originator of CNN network structure, which defines CNN network
architecture for the first time. In 2012, Krizhevsky and Hinton launched
AlexNet [71]. For the first time, AlexNet applied ReLu and dropout in CNN to
avoid overfitting the model, and its network architecture is deeper than that
of LeNet. In the ILSVRC (Imagenet Large-scale Visual Recognition Challenge) of
that year, the accuracy rate of AlexNet exceeded second place by 10.9%, which
caused many scholars to study deep learning and officially opened the prelude
of deep learning. The network structure diagram of LeNet is shown in Figure.
6.

Figure 6: The network structure diagram of LeNet. The figure matches to Fig.2
in [90].

In 2014, Simonyan and Zisserman proposed the VGG series model, which appeared
on ILSVRC2014 as the second-ranked basic network for classification tasks and
the first for positioning tasks. For the time, VGG belonged to a deeper
network, and it can be seen as a deepened version of AlexNet. VGG is composed
of 5 layers of convolutional layers, 3 layers of fully convolutional layers,
and a softmax output layer. The layers are separated by max-pooling, and the
activation units of all hidden layers use the ReLu function. The pooling core
is smaller, the number of channels is more, and the number of layers is deeper
[109].

In the same year, Google proposed the Inception network structure and built
GoogLeNet (also known as Inception Net V1) [120] based on this. This
architecture won first place in the classification and detection task of
ILSVRC2014. Unlike VGG, Inception’s structure is deeper and wider. The main
innovation is to choose to connect the convolutional layer with the
convolution kernel size of $1\texttimes 1$, $3\texttimes 3$, $5\texttimes 5$
and the pooling layer with the pooling kernel size of 3×3 in parallel, and
perform the feature maps obtained by each concatenate operations are merged as
subsequent input. This method greatly improves the efficiency of
classification. Next, in 2015, Google proposed the Batch Normalization
operation, added it to GoogLeNet and modified a certain structure, and got
Inception Net V2 [60]. Then, Inception Net V3 [121] and Inception Net V4 [119]
are successively proposed. The Inception module with dimensionality reduction
is shown in Figure. 7.

Figure 7: The Inception module with dimensionality reduction. The figure
matches Fig.2 in [120].

In 2015, Kaiming He proposed ResNet. The Residual Block in ResNet solves the
problem of poor fitting ability of deep neural networks (DNNs) after reaching
a certain depth and also alleviates the problem of gradient disappearance. The
Figure. 8 is the residual structure. The original function $H(X)$ to be fitted
is changed to $F(X)$, where $H(x)=F(x)+X$. If the depth of the model continues
to increase, as shown in Figure. 9, the original residual block is improved to
bottleneck block to reduce the parameters and calculation of the model [55].

Figure 8: Residual learning: a building block. This figure matches Fig.2 in
[55].

Figure 9: Left: a building block. Right: a bottleneck building block for
ResNet-50/101/152. This figure corresponds to Fig.5 in [55].

##  3 Datasets and Evaluation Methods

In this section, we have discussed some commonly used datasets and evaluation
metrics for the classification, segmentation, and detection tasks.

###  3.1 Publicly Available Datasets about WSI

To make better use of WSI technology to analyze CAD, we summarize some common
open datasets in our research. Tab. 1 shows the basic information for these
datasets. The two most commonly used datasets are The Cancer Genome Atlas
(TCGA) [4] and the Camelyon datasets [80]. These two datasets are usually used
for classification and detection. Meanwhile, we notice two WSI datasets named
TUPAC16 [125] and Kimia Path24 [5]. TUPAC dataset is generally used, like
mitosis detection, breast tumor proliferation prediction, automatic scoring
(classification), etc. Kimia Path24 is regularly used for classification and
retrieval [5] [72]. The basic information of the common datasets is shown in
Table. 1.

Table 1: The basic information of the publicly available used datasets. Databases | Year | Field | Number of images or or size
---|---|---|---
TCGA | 2006 | Cancer related | Over 470 TB
NLST Pathology Images | 2009 | Lung | Around 1250 H&E slides
BreakHis | 2015 | Breast cancer | 9,109 microscopic images
TUPAC16 | 2016 | Tumor mitosis | Around 821 H&E slides
Camelyon | 2017 | Breast cancer | Around 3TB
Kimia Path24 | 2017 | Pathology Images | 24 WSIs

####  3.1.1 TCGA database

TCGA is a project jointly launched by the National Cancer Institute (NCI) and
the National Human Genome Research Institute (NHGRI) in 2006 [4]. It contains
clinical data, genome variation, mRNA expression, miRNA expression,
methylation, and other data of various human cancers (including subtypes of
tumors). The database is designed to apply high-throughput genomic analysis
approaches to assist people in developing a better considerate of cancer and
improve the ability to prevent, diagnose, and treat [124]. While TCGA
fundamental task concentrates on genomics and clinical data, it also acquires
a large number of WSIs in patient’s tissue. Since WSI datasets are much larger
than other datasets, to speed viewing, David et al. [51] propose a combined
network platform named Cancer Digital Slide Archive (CDSA) to contain all WSIs
in TCGA. Because the dataset contains many types of cancers, it has a deep
range of applications. Fig. 10 is an instance of WSIs in an adrenal cortical
carcinoma in the TCGA database.

Figure 10: An example of WSI in an adrenal cortical carcinoma in the TCGA
database [4].

####  3.1.2 Camelyon Database

The Camelyon Challenge is hosted by International Symposium on Biomedical
Imaging (ISBI) [80]. The whole competition dataset (Camelyon16, Camelyon17)
are derived from sentinel lymph nodes of breast cancer patients contains WSIs
of Hematoxylin and Eosin (H&E) stained node sections [10][7].

Accordingly, the Camelyon dataset is fitting for the automatic detection and
classification of breast cancer in WSI. The data of Camelyon16 are from the
Radboud University Medical Centre (RUMC) and the University of Utrecht Medical
Centre (UMCU). The Camelyon16 dataset is composed of 170 phase I lymph node
WSIs (100 normals and 70 metastatic) and 100 Phase II WSIs (60 normals and 40
metastatic), and the test dataset includes 130 WSIs from two universities. The
Camelyon16 dataset is used as training values for the evaluation of
Camelyon17. Fig. 11 is a pathological image of a lymph node in Camelyon. The
left side refers to normal cell tissue, and the right cell has been engulfed
and held by cancer cells.

Figure 11: A pathological image of a lymph node in Camelyon [80]. The left
side belongs to normal cell tissue, and the right cell has been swallowed and
occupied by cancer cells.

####  3.1.3 TUPAC16 Database

The TUPAC16 challenge is held in the background of the MICCAI [125]. TUPAC16
main challenge dataset includes 821 TCGA WSIs with two kinds of tumor
proliferation data. 500 for training, 321 for testing. In addition to the main
challenge dataset, there are two extra datasets (area of interest and mitotic
detection). The area of interest secondary dataset involves 148 cases that are
inconsistently picked out from the training dataset. The mitotic test dataset
involved WSIs of 73 breast cancer WSIs from three pathological centers. Of the
73 cases, 23 are AMIDA13 challenge [126]. The resting 50 cases previously used
to assess the interobserver agreement for mitosis counting are from two other
pathology centers in the Netherlands. Therefore, datasets are mainly used for
automatic detection of mitosis or other regions of interest (ROI) in tumors.
Fig. 12 shows some examples of mitosis maps in H&E breast cancer slices, with
green arrows indicating mitosis.

Figure 12: Some examples of mitosis diagrams in H&E breast cancer slices in
the TUPAC16 [125], with green arrows marking mitosis.

####  3.1.4 Kimia Path24 Database

The dataset is artificially selected consciously. There are 350 WSIs from
different body positions, so 24 WSIs represent different texture patterns.
Therefore, this dataset is more like a computer vision dataset (opposite to
the pathological dataset), because the visual attention is focused on the
diversity of patterns, rather than on the medical field [5]. Therefore, the
dataset is mainly used for the classification and retrieval of
histopathological images. The 24 WSIs miniatures in the dataset are shown in
Fig. 13.

Figure 13: 24 WSIs thumbnails in Kimia Path24 database [5].

###  3.2 Evaluation Method

This subsection introduces the evaluation methods of classification,
segmentation, and detection algorithms and related formulas.

####  3.2.1 Basic Evaluation Indexs

The confusion matrix is used to observe the performance of the model in each
category, and the probability of each category can be calculated. The specific
style of the confusion matrix is shown in Tab. 2.

Table 2: Confusion matrix of basic evaluation indexs. Data Class |  Classified as Pos |  Classified as Neg
---|---|---
Pos | True Positive(TP) | False Negative(FN)
Neg | False positive(FP) | True Negative(TN)

According to the confusion matrix, the True Positive Rate (TPR) can be defined
as $TP/(TP+FN)$, which represents the proportion of the actual positive
instances in all positive instances of the positive class predicted by the
classifier, the False Postive Rate (FPR) can be defined as $FP/(FP+TN)$, which
represents the proportion of the actual negative instances in all negative
instances of the positive class predicted by the classifier. It can be seen
that the mathematical expressions of the following evaluation metrics are
shown in Table. 3 [112].

Table 3: Evaluation metrics. _Acc_ , _P_ , _R_ , _Se_ , _Sp_ and _F1_ denote accuracy, precision, recall, sensitivity, specificity and F1 score, respectively. Assessments | Formula | Assessments | Formula
---|---|---|---
_Acc_ | $\frac{TP+TN}{TP+TN+FP+FN}$ | _Se_ | $\frac{TP}{TP+FN}$
_P_ | $\frac{TP}{TP+FP}$ | _Sp_ | $\frac{TN}{TN+FP}$
_R_ | $\frac{TP}{TP+FN}$ | _F1_ | $2\texttimes\frac{P\texttimes R}{P+R}$

####  3.2.2 Evaluation of Segmentation Methods

Image segmentation [54] is the segmentation of images with existing targets
and precise boundaries. The commonly used indicators are accuracy, precision,
recall, F-measure, sensitivity, and specificity. These metrics we have
discussed in Sec. 3.2.1 and their mathematical expressions are given Tab. 2.
Dice co-efficient (D) and Jaccard index (J) are popular segmentation
evaluation indexes in recent years. Dice co-efficient (D) represents the ratio
of the area intersected by two individuals to the total area, that is, the
similarity between the ground truth and the segmentation result graph. If the
segmentation is perfect, the value is 1. Then, if S stands for the
segmentation result graph and G stands for ground truth, the expression of the
Dice co-efficient (D) is given in Eq. (1).

$D(S,G)=\frac{2|A\cap G|}{|A|+|G|}$ (1)

Jaccard Index (J) represents the intersection ratio of two individuals, which
is similar to Dice co-efficient. The formula is given in Eq. (2).

$J(S,G)=\frac{|A\cap G|}{|A\cup G|}$ (2)

####  3.2.3 Evaluation of Classification Methods

Classification [66] is the operation of determining the properties of objects
in the image. In the field of digital histopathology we studied, some are the
classification of cancer [97], some are the operation of selecting ROI [118],
and some are the identification of cancer regions [35]. The purpose of
classification is achieved by the constructed classifier. The performance
indicators used to evaluate these classifiers are critical to the final
results. Accuracy is the most commonly used indicator to evaluate classifiers.
Precision, recall, sensitivity, specificity, and F1 score are widely used to
evaluate classifiers. Accuracy,precision, recall, F-measure, sensitivity, and
specificity we have discussed in Sec. 3.2.1, and their mathematical
expressions are given in Tab. 2. With the continuous improvement of
classification requirements in practical applications, ROC (Receiver Operating
Characteristic), AUC (Area Under ROC Curve), a non-traditional measurement
standard, has emerged. ROC is a curve drawn on a two-dimensional plane with
FPR as the abscissa and TPR as the ordinate. It can reflect the sensitivity
and specificity of the continuous variables as a comprehensive indicator. It
can also solve the problem of class imbalance in the actual dataset. AUC
quantifies the area under the ROC curve into a numerical value to make the
results more intuitive.

####  3.2.4 Evaluation of Detection Methods

Detection [94] is another common task in analyzing histopathological WSIs.
Detection is not only to determine the attributes of the region identified in
WSI but also to identify and obtain more detailed results. Because of the
similarity between testing and classification, most of the evaluation indexes
are the same as the classification, including accuracy, precision, recall,
F-measure, sensitivity, and specificity that we have discussed in Sec. 3.2.1.
However, in WSI detection, it is difficult to locate, determine and quantify
multiple lesions. Therefore, FROC (Free Receiver Operating Characteristic
Curve) [37] is proposed to evaluate the detection results. FROC curve is a
small variation of the ROC curve. It is a curve drawn on a two-dimensional
plane with FP as the horizontal coordinate and TPR as the vertical coordinate.
This allows the detection of multiple lesion areas on a single WSI.

###  3.3 Summary

According to the review above, we can see that the commonly used public
datasets are TCGA, TUPAC16, and Kimia Path24 for the classification,
segmentation, and detection of histopathological images using the combination
of WSI technology and CAD with a brief introduction. Also, the evaluation
indicators of these three tasks. The basic commonly used evaluation indicators
are accuracy, precision, recall, sensitivity, and specificity. In terms of
classification, there are comprehensive indicators such as AUC. Dice co-
efficient and Jaccard index in segmentation indicators have become popular in
recent years, and FROC in detection indicators can be used for positioning,
qualitative, and quantitative analysis of multiple lesions.

##  4 Traditional Neural Network Method

As can be seen from Figure. 2, ANN is separated into two parts. It is a
shallow traditional neural network and a DNN. In the content, we investigated,
traditional shallow neural networks are separated into Perceptrons, Radial
Basis Function (RBF), Back Propagation Neural Networks, etc. The DNN is
generally separated into CNN, Recurrent Neural Network (RNN) and Generative
Adversarial Network (GAN) and so on.

In the papers we summarized earlier in the year, there were cases where
shallow traditional neural networks were applied to WSI for CAD. In [97], the
LNKNET software package is used to classify breast cancer WSI. LNKNET
integrates neural networks, machine learning, classification and statistics,
clustering, and feature selection algorithms into a modular package to achieve
the purpose of classification.

In [90], an automatic feature learning method for WSI classification from
unlabeled images is proposed. The first step is to learn the dictionary from
unlabeled images. Then the sparse auto-encoder is used to learn the function.
The automatic encoder consists of three parts: Encoder, dictionary, and a set
of codes. The encoder is used to train the classifier on a small amount of
label data. The multi-class regularization support vector classification is
used. The regularization parameter is 1 and the polynomial kernel is 3. For
data, two datasets from TCGA (i) glioblastoma multiforme (GBM) and (ii) clear
cell renal cell carcinoma (KIRC) were used. The accuracy of classification is
$84.3\%$ and $80.9\%$ respectively. The classification results of uneven GBM
tissue sections are shown in the Figure. 14.

Figure 14: Two examples of classification results of a heterogeneous GBM
tissue sections. The figure matches Fig.4 in [90].

##  5 Deep Neural Network Method

###  5.1 Segmentation Methods

Among the papers we reviewed, 10 papers used deep learning methods for WSI
segmentation [8, 135, 33, 24, 110, 85, 102, 103, 40].

In [8], FCN and U-net are used to segment and accurately identify tissue
slices. In this paper, the two methods are compared with the traditional
foreground extraction (FESI) algorithm based on structural information.
Finally, the three methods are applied to 54 WSIs and evaluated using the
average value of the Yakoka index and its standard deviation. The final U-net
results are the best (Jaccard index is $0.937$). Figure. 15 shows the
qualitative effects of different algorithms.

Figure 15: Qualitative results for the different algorithms. The figure
matches to Fig.2 in [8]

In [135], CNN-based Imagenet is used to extract features and transform the
extracted features into histopathological images. Support vector machine (SVM)
is used to define segmentation problems as classification problems. Finally,
the method is applied to the digital pathology and colon cancer dataset of
MICCAI 2014 challenge, and the final result win first place with $84\%$
accuracy.

In [33], an efficient and simple framework called Reinforced Auto-Zoom Net
(RAZN) is proposed, which considers the accurate and fast prediction of breast
cancer segmentation. RAZN learning strategy network to decide whether to
enlarge a given ROI. Because the amplification is selective, RAZN is robust to
unbalanced and noisy ground truth tags and effectively reduces overfitting.
Finally, the method is evaluated on a public breast cancer dataset. It can be
seen from the experimental results that RAZN is superior to single-scale and
multi-scale baseline methods, and can obtain higher accuracy with lower cost.

In [24], an automatic end-to-end DNN algorithm is proposed to segment the
single core. The core boundary model is introduced to predict the core and its
boundary simultaneously by FCN. Given the normalized color image, the model
outputs the estimated kernel and boundary map directly. Then after the post-
processing, the core of the final segmentation is produced. A method for
extracting and assembling overlapping blocks is designed to predict the kernel
in large WSI seamlessly. Finally, the results show that the data extension
method is effective for the task of nuclear segmentation. The experiment shows
that the method is superior to the existing technology and can segment WSI
accurately in the acceptable time range.

In [110], different architectures are systematically compared to evaluate how
multi-scale information can affect segmentation performance. The architecture
is shown in Figure. 16. The experiment used a public breast cancer dataset and
a locally collected prostate cancer dataset. The final results show that
visual environment and scale play an important role in histological image
classification.

Figure 16: Used architectures. The figure matches to Fig.2 in [110]

In [85], a new encoder-decoder architecture is proposed to deal with the
semantic segmentation problem of breast biopsy WSI. The designed new
architecture includes four new functions: (1) Input Perceptual Coding Block
(IA-RCU), which can enhance the internal input of the encoder to compensate
for information loss caused by downsampling, (2) densely connected decoding
network, and (3) additional sparse connection of the decoding network to
effectively combine the multi-level features of the encoder aggregation, and
(4) a multi-resolution network for context-aware learning, which uses densely
connected fusion blocks to combine rate outputs of different resolutions. The
result after segmentation is shown in Figure. 17.

Figure 17: Segmentation result. The figure corresponds to Fig.9 in [85].

In [102] and [103], several U-net architectures and DCNN, which are used to
output probability maps, are trained to segment the WSI of ductal carcinoma in
situ (DCIS) in wireless sensor networks, and the minimum required level of a
good patch vision for achieving excellent accuracy on slides is verified.
U-net is trained five times to achieve the best test results ($DSC=0.771$,
$F1=0.601$), which means that U-net can benefit from broader context
information. The U-net structure used is shown in the Figure. 18.

Figure 18: U-net structure used in [103]. The figure corresponds to Fig.2 in
[103].

In [40], a multi-scale image processing method is proposed to solve the image
segmentation problem, that is, the localization problem in the
histopathological imaging of liver cancer. These eight networks are compared,
and the most suitable network for liver cancer segmentation is obtained.
Through comprehensive performance comparison, U-net is selected. The local
color normalization method of the pathological image is used to solve the
influence of the background, and then a seven-layer Gaussian pyramid is used
to represent it. Multi-scale image sets are then created for each WSI. Well-
trained U-net, fine-tuned at each level to obtain independent models. Then,
shift cropping uses weighted overlap in the prediction process to solve the
problem of block continuity. Finally, the predicted image is mapped back to
the original size, and voting is performed, and a multi-scale predicted image
fusion mechanism is proposed. The experimental data is the verification image
of the 2019 MICCAI PAIP Challenge. This evaluation result shows that the
algorithm is better than other algorithms.

Table. 4 summarizes the work of different teams using ANNs to analyze WSI for
segmentation tasks.

Table 4: Summary of WSI segmentation tasks by different teams using artificial neural network (ANN). Year |  Reference |  Team |  Method |  Evaluation Index
---|---|---|---|---
2017 |  [8] |  Bándi P |  FCNN, U-Net |  Jaccard index
2017 |  [135] |  Xu Y |  SVM-CNN |  Accuracy
2018 |  [24] |  Cui Y |  FCN |  |  precision, recall, F1,
---
Dice co-efficient
2018 |  [110] |  K Sirinukunwattana |  CNN |  |  Classification accuracy as
---
measured by the F1-measure
2018 |  [85] |  Sachin Mehta |  |  multi-resolution
---
encoder-decoder network
F1, mIOU, PA
2018 |  [33] |  Nanqing Dong |  |  Reinforced Auto-Zoom Net
---
(RAZN)
|  intersection over union (IOU),
---
Relative inference time
2019 |  [102] |  Nikhil Seth |  |  original U-Net architecture
---
to use ELU and
a multi-resolution network,
Ensemble models
|  Dice Score, Dice P-Value,
---
F1 Score, F1 P-Value, Time(s)
2019 |  [103] |  Seth N, Akbar S |  U-Net |  Dice co-efficient, Modified F1 Score
2020 |  [40] |  Feng Y, Hafiane A |  U-net |  |  F1 score, Jaccard similarity score
---
and directed Hausdorff distance

###  5.2 Classification Methods

In this section, the relevant contents of using deep learning algorithms to
classify histopathological WSIs are summarized.

Metastatic breast cancer needs to be identified in [127]. Firstly, the
background is detected automatically based on the threshold. By comparing the
performance of GoogLeNet, AlexNet, VGG16 and FaceNet, GoogLeNet is selected to
generate tumor probability thermogram and RF classifier is used to classify
metastatic WSI and negative WSI, and finally AUC of $0.9254$ is obtained.

In [107], the complete convolution network is used to automatically label the
molecular biomarkers of the complete IHC image. First, the RGB image is input
into FCN, and the output is the image of different biomarkers. Each cell is
classified according to the biomarkers it expressed. The results show that the
proposed method is more effective than the manual marking method (the average
F-score is 0.96). The neural network training process is divided into two
parts: (1) Training CNN to label biomarkers expressed in cells; (2) converting
the trained CNN into FCN to locate all cells expressing different biomarkers
in WSI. The purpose of this paper is to train CNN through the kernel images
separated from WSI and transform the trained CNN into end-to-end pixel to
pixel networks like FCN, which can locate WSI biomarkers and output heat maps
expressed by different biomarkers.

In [62], normal sections and tumor sections are classified from histological
images of lymph node tissues. The first step is to delete the useless
information. Then the CNN model is reconstructed or trained to segment the
tumor region. The details are in Sec. 5.3. After the tumor region is
segmented, features are extracted and RF classifier is used for
classification. Finally, the AUC score is $0.94$.

In [3], WSIs of basal cell carcinoma are studied as a complete unsupervised
feature. First, there is a set of feature detectors. This feature detector is
selected from a set of patches randomly selected from the collection of
images. Then this group of feature detectors is used to capture the most
common patterns, and the method used is to simulate an auto-encoder neural
network. Then convolution or BOF method is used to represent the image. This
representation is implemented using the feature detector learned in the
previous step. Next, the binary classification model, the softmax regression
classifier, is trained from the representation obtained by the convolution or
BOF method. Basal cell carcinoma includes different types of cancer and non-
cancer. The final result of the system is shown in Figure. 19. The best result
is obtained in AUC, which is better than the most advanced $7\%$ and $98.1\%$.

Figure 19: Outputs produced by the system for different cancer and non-cancer
input images. The figure matches to Table.2 in [3].

In [44] and [45], a deep convolution network is used to detect and classify
WSI and ROI in breast cancer. The detection part is described in detail in
Sec. 5.3. This part mainly introduces the classification, that is, ROI
includes five diagnostic categories. A classifier is designed based on CNN,
and the features obtained by CNN are used to classify the detected ROI. The
structure of CNN is shown in Figure. 20. Then the WSI classification is
processed. WSI classifies according to the predictions of most categories in
the remaining cancer areas. The results show that the efficiency is increased
about 6.6 times with enough accuracy.

Figure 20: Designs of the CNN. The figure matches to Figure.4.6 in [44].

In [111], it mainly includes two parts, namely the detection and
classification of nuclei. The detection part proposes a space-constrained CNN
for nuclear detection. The details are in Sec. 5.3. The classification uses a
combination of CNN and the new NEP. None of the proposed detection and
classification methods require kernel segmentation. Then the method is applied
to colorectal adenocarcinoma WSIs, and the final classification result
obtained a higher F1 score than the traditional method.

In [56], an expectation-maximization method is proposed. The spatial
relationship of patches is used to locate the distinguishable patches
accurately. This method is suitable for the subtype classification of glioma
and non-small cell lung cancer. The classification module uses CNN at patch-
level and trains the decision fusion model into a two-level model. The first
level (patch-level) model is based on expectation-maximization and combines
the prediction of CNN output patch-level. In the second level (image-level),
block-level prediction histogram is input to image-level multiple logistic
regression or SVM.

In [5], a new dataset, Kimia Path24, is introduced for image classification
and retrieval in digital pathology. The WSIs of 24 different textures are
generated to test patches. Especially, the patch classification is based on
LBP histograms, bag-of-words approach, and CNN.

In [2], a WSI classification method using CNN for breast biopsy is proposed.
The proposed network architecture can retrieve information of different
scales. First, the image contains 12 non-overlapping color blocks, and then
the CNN and CNN + SVM classifiers trained by segments are used to calculate
the probability of color block level. Finally, one of three different patch
probability fusion methods is used to obtain the image classification results.
The three methods are majority voting (select the most common patch as the
image tag), maximum probability (select the patch category with high
probability as the image tag), and probability (the category with the largest
sum of patch-level probability). The final results divided the images into
four categories: Invasive, in situ, benign and normal. The proposed system
achieves an overall sensitivity of about $81\%$ for cancer patch
classification.

In [11], a context-aware stacked CNN Is proposed to classify WSI as
normal/benign, DCIS and invasive ductal carcinoma (IDC). The first step is to
train CNN and capture cell-level information with high pixel resolution. Then,
the feature response generated by the model is input to the second CNN and
superimposed on the first CNN. The AUC of binary classification of non-
malignant and malignant slides is $96.2\%$, and the three-level accuracy of
normal/benign classification of WSI, DCIS and IDC are $81.3\%$.

In [105], the automatic classification of gastric cancer WSI is proposed and
compared with the traditional image analysis method and deep learning method.
In the traditional analysis method, GLCM, Gabor filter, LBP histogram, gray
histogram, HSV histogram and RGB histogram are used for classification and RF.
In terms of deep learning methods, AlexNet is proposed as a deep convolution
framework. The network structure is shown in Figure. 21. According to the
experiment, the overall classification accuracy of the cancer classification
proposed by AlexNet is $0.6990$, and the overall classification accuracy of
necrosis detection is $0.8144$.

Figure 21: Proposed AlexNet architecture. The figure matches to Figure.4 in
[105].

In [69], different types of colorectal polyps on WSIs are classified to help
pathologists diagnose them. The architecture used is an improved version of
the ResNet structure. WSI is divided into several patches and then applied to
ResNet. If at least five patches on WSI are identified as such, the average
confidence is $70\%$. If there is no cancer type in the patch, the WSI is
considered normal. Finally, the accuracy is $93.0\%$. The recall is $88.3\%$;
F1 scores is $88.8\%$.

In [58], a DNN (an 11-layer CNN model) is trained to automatically learn
effective features and classify protein images of 8 subcellular locations.
Firstly, image preprocessing and data balance are carried out. Then, the
processed data is passing through the 11 layers of the CNN model. The details
of the structure are shown in Figure. 22. Finally, the classification accuracy
of test data is $47.31\%$, and the classification accuracy of training data is
$100\%$.

Figure 22: The network architecture diagram and the visualization of the
middle layer in [58]. The figure matches to Figure.4 in [58].

In [135], a deep CNN is proposed to classify, segment, and visualize large-
scale histopathological images. In the part of the classification, firstly,
WSI is divided into patches, then the background is discarded, and then the
selected patches are input into the network to obtain a 4096-dimensional CNN
feature vector. The final feature vectors are combined by softmax. Function
selection is then performed to remove redundant and irrelevant functions.
Finally, SVM is used for classification and the MICCAI challenge dataset is
used for the experiment. The accuracy of classification is $97.5\%$.

In [70], an image analysis method based on deep learning is proposed to
classify different polyp types on WSI. The classification model is based on
ResNet architecture, with minor modifications. Specifically, the last fully
connected layer is replaced by a convolution layer. The final result is that
the average accuracy is $91.3\%$, which is better than other deep learning
architectures.

In [46], a new deep learning method is proposed to classify white blood cells
in WSI. The network uses the average pool level to find the hot spots of white
blood cells in WSI. The first is to fine-tune the pre-trained AlexNet
according to the dataset. Then, the network is trained on the patch dataset,
and the trained network structure is shown in Figure. 23

Figure 23: Architecture of the DCN used in approach. The figure matches to
Figure.1 in [46].

In [27], a network structure based on CNN is proposed. The tissue sections of
WSI are analyzed by various resolution methods. That is to say, the class a
posteriori estimation of each view at a specific magnification is obtained
from CNN at a specific magnification, and then the class a posteriori
estimation of random multiple views at multiple magnifications is voted and
filtered to provide a slide-level diagnosis. According to the experimental
results, the final classification accuracy is $94.67\pm 14.60\%$, sensitivity
$96.00\pm 8.94\%$, specificity $92.00\pm 17.85\%$, and F-score of $96.24\pm
5.29\%$.

In [98], prostate histopathology WSI is graded. The main method is an
unsupervised domain adaptive method. The adaptation here is achieved through
confrontation training, which can minimize the distribution difference of
feature space between two domains (annotated source domain and untagged target
domain) with the same number of high-level classes. The loss function also
uses Generative Adversarial Network (GAN). Besides, a Siamese architecture is
developed to standardize patches in WSI. The method is then applied to the
public prostate dataset for verification. The experimental results show that
this method significantly improves the classification accuracy of the Gleason
score.

In [19], a method of classification and localization of diseases with weak
supervision is proposed. First, WSIs pretreatments are performed, including
foreground and background segmentation, color normalization, and tile
partitioning. Then ResNet is used for feature extraction. Then, the WELDON
method is proposed by Durand et al. is improved and adjusted [36].
Modification of the pre-trained deep CNN model, feature embedding, and
introduction of an additional set of full connection layers for context re-
classification from instances. The final classification output diagram is
generated as shown in Figure. 24.

Figure 24: The classification result of the proposed method in Camelyon16
dataset. The figure matches to Figure.4 in [19].

In [123], a two-part method is proposed to classify WSI. Firstly, the encoder
is trained in an unsupervised way, the organization block on the WSI is mapped
to the embedded vector, and the stack of the WSI feature graph is formed by
using the sliding window. Then, the CNN classifier is trained based on WSI
compact representation. There are three types of encoders trained here: A
convolutional automatic encoder (CAE), a variational automatic encoder (VAE),
and a new method based on contrast training. Experiments show that the new
contrast encoder is better than CAE and VAE.

In [74], a multi-classification of breast cancer in WSI is presented. They
propose a deep learning framework, which is mainly divided into two stages,
using microscopic images and WSI to achieve the purpose of classification.
After the two types of images are patched, the microscopic images are used for
Inception-ResNet-v2 to train the classifier. The WSIs are then subsampled and
converted from RGB to CIE-LAB color space and segments the foreground from and
background. Then the extraction of hard examples and patch classifier is
retrained. The prediction results are aggregated from the block by block
prediction back to image prediction and WSI annotation. This method is applied
to ICIAR 2018 Grand Challenge on breast cancer histology images, with an
accuracy rate of $87\%$, far exceeding second place. The specific working
process is shown in Figure. 25.

Figure 25: Overview of the framework. The figure matches to Figure.1 in [74].

The author in [26] also proposes a multiple instance learning (MIL) framework
based on breast cancer WSI. The framework is based on CNN and introduces a new
pooling layer, which enables patches in WSI to aggregate most information
functions. This pooling layer is a new layer of multi-instance pool (MIP),
which introduces MIL into DNN as an end-to-end learning process to realize WSI
classification. Finally, high classification sensitivities of $93.87\%$,
$95.81\%$, $93.17\%$, and $88.95\%$ are achieved with four different
magnifications using the public dataset.

The study in [16] is also based on the MIL classification for prostate cancer.
Slide tiling is the first time to run at different magnification and generate
its package. Then, the model is trained to find the tile block with the
highest positive and negative probability in the slide, which is used to pay
more attention to the less representative examples.Based on the classification
of AlexNet, ResNet, and VGG, given the threshold, if at least one instance is
positive, then WSI is called positive, if all instances are negative, then the
slide is negative. The optimal models are ResNet34 and VGG11-BN, with AUC of
$0.976$ and $0.977$, respectively. The method of [15] is roughly the same as
that of [16].

[137] is used to automatically evaluate the classification accuracy of gastric
cancer biopsy image analysis system (E-pathologist).

In [130], the WSI lung cancer is classified by weakly supervised learning.
Firstly, the improved FCN algorithm based on patch-level of WSI is applied to
cancer prediction to find different regions as patch-level prediction models.
When the probability of a block exceeds the threshold, it is retrieved. Then,
context-based feature selection is performed and aggregated from the retrieved
parts. The global feature descriptor is constructed. Finally, the global
feature descriptor is input into the standard RF classifier. Finally, a high
classification accuracy of $97.1\%$ is obtained. The whole proposed method is
shown in Figure. 26

Figure 26: The whole proposed method. The figure matches to Figure.1 in [130].

.

In [108], CNN is mainly used to classify gastric cancer in WSI. Firstly, the
difference of the threshold value and color feature is used to extract the
tissue and conduct morphological processing. Then, it is separated into
patches, and the data is expanded by flipping. Then, the existing CNN
architecture is utilized to conduct experiments on patch-level and slide-
level. Finally, good results are obtained on DenseNet-201. And [137] is used
to automatically evaluate the classification accuracy of gastric cancer biopsy
image analysis system (E-pathologist).

In [91], the WSI of the patient is down-sampled to five times magnification,
and the ROI is extracted according to the Otsu threshold and the manually
refined tissue boundary. Then train the classifier to classify whether the
patient has heart failure. Among them, two classifiers are trained. The first
uses CNN, and the second uses traditional feature engineering methods based on
WND-CHARM [92]. It can be seen from the experimental results that CNN is
better than WND-CHARM in terms of classification accuracy and sensitivity. It
shows that cardiac histopathology is sufficient to accurately identify
patients with clinical heart failure.

In [77], the prostate WSI is classified by multi-resolution based on attention
mechanism. The attention-based MIL model is used to extract transient
features. The whole process is divided into two parts. In the first part,
attention-based clustering block selection is used to classify cancer and non-
cancer. The second part is the research of cancer classification with higher
resolution. Finally, the average classification accuracy is $85.11\%$, which
realizes the new performance of prostate cancer classification.

In [138], a machine learning algorithm for predicting the prognosis of
colorectal cancer from WSI is introduced. First, data preprocessing is
performed, including chroma normalization, color block extraction, and data
enhancement. Then the classification process of unsupervised learning is
carried out. The $K$-means algorithm first uses grouping to group patches from
a single WSI, and then uses different random initial parameters for multiple
clustering (two clustering methods are used here: Information density
clustering and phenotype clustering). Then, CNN-based classifiers are used to
train patches from different clusters. Finally, SVM is used to learn the
cluster-level results. Finally, the results are used to predict the prognosis
of patients.

In [133], a Deeptissue Net is used to segment WSIs of colorectal cancer
histopathology images. The whole process is shown in Figure. 27. (a-f) is the
training process of the model. The 10 tissue sections are then marked at (a),
and generate training image blocks (e) from (b) for training Deeptissue Net
(f). Then test from (g) to (j) and compare the final result with manual
annotation (k), and evaluate the performance through the confusion matrix.
Experiments show that the classification performance of Deeptissue Net is
significantly better than ResNet and DenseNet, reaching a classification
accuracy of $96\%$.

Figure 27: WSI Organization Classification Deep Organization Network Diagram.
The figure matches to Figure.2 in [133].

.

In [129], multi-instance deep learning is also used to classify WSI gastric
cancer images. A method named the RMDL method is proposed. Similar to the
above MIL method, it is separated into two stages. In the first stage,
distinguishable instances are selected, and the method is to train the
localization network. In the second part, the local-global features are fused,
and the RMDL network is combined by instance recalibration and a multi-
instance pool module, which is used for image label prediction. The specific
working process is shown in Figure. 28.

Figure 28: Overview of the framework. The figure matches to Figure.2 in [129].

In [113], a new image analysis toolbox is proposed for the diagnosis of
cervical intraepithelial neoplasia (CIN) on the WSI of cervical tissue
samples. The inside of the toolbox consists of four parts. The first is the
preprocessing step of epithelium detection to extract the ROI. Then a binary
classifier is used to filter non-epithelial regions of high-resolution
interest. Then the epithelial cells are segmented using the EpithNet64 model
[114], and preprocessed by threshold, morphology, and smoothing filter. Local
cell growth pattern analysis is then performed by extracting small vertical
sections. Finally, the DeepCIN model [115] is used for CIN classification.
Finally, they are integrated into an image analysis toolbox.

Table. 5 summarizes the work of different teams using ANNs to analyze WSI for
classification tasks.

Table 5: Summary of WSI classification tasks by different teams using artificial neural network (ANN). Year |  Reference |  Team |  Method |  Evaluation Index
---|---|---|---|---
2014 |  [20] |  Angel Cruz-Roa |  CNN, Random Forest Classifier |  |  TP, FP, TN, FN, Precision,
---
Recall, Sensitivity,
Specificity, F1, BAC
2015 |  [3] |  John Arevalo Angel Cruz-Roa |  $K$-means |  AUC
2016 |  [104] |  Sharma H |  CNN |  $\backslash$
2016 |  [56] |  Hou L |  CNN |  $\backslash$
2016 |  [127] |  Dayong Wang Aditya Khosla |  |  a random forest classifier,
---
GoogLeNet
AUC, FROC
2016 |  [111] |  Sirinukunwattana K |  |  Neighboring Ensemble Predictor
---
(NEP) coupled with CNN
Weighted Average F1 score
2016 |  [107] |  Fahime Sheikhzadeh |  FCN |  Accuracy and average F-score
2016 |  [44] |  Baris Gecer |  FCN, CNN |  Accuracy
2017 |  [2] |  Araújo T |  |  patch-wise trained CNN
---
and CNN+SVM classifiers
Sensitivity and Accuracy
2017 |  [11] |  Bejnordi B E |  context-aware stacked CNN |  AUC, Accuracy
2017 |  [69] |  Korbar B |  a modified version of a ResNet |  |  Accuracy, precision, recall,
---
and F1 score and their
95% confidence intervals
2017 |  [58] |  Hu J X |  CNN |  Accuracy
2017 |  [27] |  Das K, Karri S P K |  |  CNN, majority voting over
---
random multi-views
at multi-magnification
$\backslash$
2017 |  [105] |  Sharma H |  self-designed CNN architecture |  Average classification accuracy
2017 |  [5] |  Babaie M |  dictionary approach and CNNs |  Accuracy
2017 |  [135] |  Xu Y |  SVM-CNN |  Accuracy
2017 |  [46] |  Ghosh A |  deep convolutional network |  |  TP, TN, FP, FN, Acc,
---
Specificity, Sensitivity, F-score
2017 |  [62] |  Jamaluddin M F |  CNN, Random Forest |  AUC
2017 |  [70] |  Korbar B |  ResNet |  Average IOU
2018 |  [98] |  Jian Ren |  A Siamese network |  $\backslash$
2018 |  [137] |  Hiroshi Yoshida |  |  used multi-instance learning
---
(MIL) for training a multilayer
neural network (MLNN) model
|  Sensitivity, Specificity,
---
Positive predictive value,
Negative predictive value
2018 |  [123] |  David Tellez |  a CNN-based classifier |  Area under the ROC curve
2018 |  [26] |  Kausik Das |  a MIL framework for CNN |  $\backslash$
2018 |  [130] |  X Wang |  |  FCN, ScanNet,
---
Random Forest (RF) classifier
$\backslash$
2018 |  [108] |  Junni Shou |  |  CNN(DenseNet-201 is the
---
best structure), RF Classifier
Accuracy, Sensitivity, Specificity
2018 |  [16] |  Campanella G |  VGG11-BN and ResNet34 |  |  Accuracy, Confusion matrix,
---
and ROC curve
2018 |  [19] |  Courtiol P |  MLP Classifier |  AUC
2018 |  [45] |  Baris Gecer |  FCN, CNN |  $\backslash$
2018 |  [74] |  Scotty Kwok |  Inception-Resnet-v2 |  $\backslash$
2018 |  [91] |  Jeffrey J. Nirschl |  CNN |  Sensitivity, specificity
2019 |  [15] |  Gabriele Campanella |  CNN and MIL |  AUC
2019 |  [12] |  S Bilaloglu |  PathCNN |  AUC
2019 |  [129] |  Shujun Wang |  |  a recalibrated multi-instance
---
deep learning method(RMDL)
Accuracy, Average classification score
2019 |  [138] |  Xingzhi Yue |  $K$-means, CNN, SVM |  Accuracy, F1-score
2019 |  [133] |  Jun Xu |  Deeptissue Net |  Accuracy
2019 |  [77] |  Jiayun Li |  Vgg11bn, $K$-means clustering |  Accuracy
2020 |  [113] |  Sudhir Sornapudi |  DeepCIN model |  P, R, F1, ACC, AUC, AP, MCC

###  5.3 Detection Methods

This part introduces some detection methods of DNN for histopathology WSIs to
improve the work efficiency of pathologists.

####  5.3.1 CNN-based Deep Learning Detection Method

The following [139, 22, 10, 122, 67, 62, 20] are all detection classifiers
based on CNN.

In [139], conditional random field (CRF) is applied to the potential space of
deep CNN after training, and the compact features extracted from the middle
layer of CNN are regarded as the observations in the fully connected CRF model
to detect invasive breast cancer. The experimental results showed that the
average FROC score of tumor area detection in histopathological WSI increased
by about $3.9\%$. The proposed model is trained on Camelyon17 ISBI Challenge
the data set and finish second with a kappa score of 0.8759.

In [22], involved images from five different cohorts from different
institutions/ pathology labs in the United States of America and TCGA. The
training dataset has 349 estrogens receptor-positive (ER+) invasive breast
cancer patients. The approach yielded a Dice-coefficient of $75.86\%$, a
positive predictive value of $71.62\%$ and a negative predictive value of
$96.77\%$ in terms of pixel-by-pixel evaluation compared to manually annotated
regions of IDC.

In [10], the authors evaluated the performance of automatic deep learning
algorithm in detecting H&E lymph node metastasis of breast cancer women, and
compared it with the diagnosis of pathologists in the diagnostic environment.
This dataset is a sample of 399 breast cancer patients from RUMC and UMCU. The
algorithm in this paper is superior to the artificial algorithm of the
pathologist.

In [122], a method for training and evaluating CNN in WSI mitosis detection of
breast cancer is proposed. In the three tasks of Tupac challenge, the
performance of the proposed method is evaluated independently.

In [67], an autofocus quality detector called ConFocus is developed to detect
and quantify the out-of-focus area and severity of WSIs. The ConvFocus
architecture resulted in Spearman ranking co-efficients of $0.81$ and $0.94$,
which are better than the results obtained by the pathologist. The expected
OOF pattern is also reproduced from Z-Stack scans in two experimental
scanners. In addition, this paper discusses the relationship between the
accuracy of state-of-the-art metastatic breast cancer monitors and OOF. It
turns out to be a negative correlation.

[62] is divided into two parts. The first part uses CNN to detect the possible
tumor location in WSI, and the second part uses the detected results to
extract features to classify normal or tumor. Using the camelyon16 dataset,
which contains 160 negatives and 110 positive WSIs for training, and 50
positives and 80 negative WSIs for testing. The AUC of this method at $0.94$
is better than that of the winner of Camelyon16 challenge at $0.925$. The CNN
structure designed in [62] is shown in Figure. 29.

Figure 29: 12 convolutional layers including the fully connected layer. This
model was also inspired by the VGG model.

In [20], CNN is trained to detect IDC in WSI. In the end, $71.80\%$ of
F-measure (F1) and $84.23\%$ of balanced accuracy are obtained. The overall
detection framework of [20] is shown in Figure. 30.

Figure 30: Overall detection framework of [20].

####  5.3.2 FCN-based deep learning detection method

The following [44, 45, 78, 79] are processed with 4-layers of FCN to achieve
the purpose of detection.

The study in [44] includes two tasks: retrieval and classification. The
retrieval part consists of four layers of FCN. Through FCN-1 feedforward
processing, obvious regions are detected from WSI, and each connected
component above the threshold is amplified on the input image, which is
processed by FCN-2. This process lasts four times and can detect an important
area of WSI. Then, CNN classifies and finally determines the diagnosis result
of breast cancer.

In [45], a WSI diagnosis system for breast biopsy is proposed. First, four
FCNs are used for saliency detection and multi-scale localization of ROI.
Then, the convolution network is used for cancer classification. Finally, the
saliency map and classification map are combined. The test results show that
there is no difference between the accuracy and the prediction of
pathologists.

In [78], an improved FCN layer is used to input the WSI of any size, and the
standard FCN layers are converted into the anchor layer. Through the anchor
layer and the fast and dense ScanNet, the network can be made faster. Its
cancer metastasis detection results show excellent performance on the
Camelyon16 challenge dataset. A similar method is applied in [78] and [79].

####  5.3.3 Other deep learning detection methods

The following references [21, 111, 12] are used for the detection after
combining CNN structure with other methods or simplifying them.

In [21], a new efficient adaptive sampling method based on probability
gradient and Quasi Monte Carlo sampling is used in combination with the CNN
classifier. It is suitable for the detection of invasive breast cancer on WSI.
The experimental results show that Dice co-efficient is $76\%$, which is an
effective strategy.

In [111], in conventional colon cancer WSI proposes a space-constrained CNN
for nuclear detection. The new NEP is used in conjunction with CNN to more
accurately predict the type of cell detection. The test result shows that this
article produces a higher average F1 score of detection.

In [12], a simplified CNN, named PathCNN, is used to detect outliers of WSI in
whole cancer. The WSI used in the experiment is downloaded from the genome
data sharing database of the TCGA.

###  5.4 Other deep learning Methods

In this part, we select the other detection methods that appear in the papers
and summarize these methods.

In [82], a new attention-based multi-mode RNN architecture, called CORAL8, is
proposed for direct renal immunofluorescence (RDIF) detection, which solves
the problem of generating medical reports for multiple image panels. Among
them, the prior encoder learns to extract the context features of doctors’
clinical notes, while VGG16 extracts the local and global features of WSIs.
Then the two are input into RNN (sentence generator) to get the final medical
report, and machine-generated text is realized. The process is shown in the
Figure. 31.

Figure 31: This image illustrates the framework of the proposed CORAL8
architecture. The figure matches to Figure.1 in [82].

In [99], the CNN architecture of LSTM is used to analyze the risk of disease
progression and recurrence in prostate cancer patients with the Gleason score
of 7. The data used in this study came from TCGA, which identified the image
biomarkers of WSI and divided the Gleason score of prostate cancer patients
into $3+4$ or $4+3$. The results show that the combination of image features
and genomic pathway score is more suitable than standard clinical markers and
image texture features for evaluating the possibility of recurrence. Patients
with the Gleason score of $4+3$ has a higher risk of progression and
recurrence than patients with Gleason score of $3+4$. The structure of the
LSTM CNN using WSI and genomic data to quantify images and genetic biomarkers
is shown in Figure. 32.

Figure 32: The architecture of convolutional neural networks (CNN) with LSTM.
The figure matches to Figure.1 in [99].

Table. 6 summarizes the work of different teams using ANNs to analyze WSI for
detection tasks.

Table 6: Summary of WSI detection tasks by different teams using ANN. Year |  Reference |  Team |  Method |  Evaluation Index
---|---|---|---|---
2014 |  [20] |  Angel Cruz-Roa |  CNN |  F-measure (F1), balanced accuracy
2016 |  [111] |  Korsuk Sirinukunwattana |  SC-CNN and NEP |  F1-score
2016 |  [44] |  Geçer B |  CNN |  ROC, accuracy
2017 |  [62] |  M F Jamaluddin |  CNN |  AUC
2017 |  [22] |  Cruz-Roa A |  ConvNet |  Dice, PPV, NPV
2017 |  [10] |  Bejnordi B E |  |  DCNN, Supervised classifiers
---
(such as SVMs, RF classifiers)
AUC
2018 |  [139] |  Farhad Ghazvinian Zanjani |  |  apply CRFs over latent spaces
---
of a trained deep CNN
FROC,kappa
2018 |  [122] |  David Tellez |  CNNs |  F1-score
2018 |  [91] |  Jeffrey J. Nirschl |  |  CNN classifier, 1000 tree
---
Breiman-style random
decision forest
(comparative approach)
|  Accuracy, Sensitivity, Specificity,
---
Positive predictive value, AUC
2018 |  [21] |  Angel Cruz-Roa |  |  High-throughput Adaptive
---
Sampling (HASHI) for
whole-slide Histopathology Image
analysis, Based on CNN,
RF and SVM
|  Dice, PPV, NPV, TPR,
---
TNR, FPR, FNR
2018 |  [45] |  Baris Gecer |  FCN |  Accuracy
2018 |  [78] |  Huangjing Lin |  |  fast and dense scanning framework,
---
referred as ScanNet (modified FCN)
FROC, AUC
2019 |  [67] |  Kohlberger T |  ConvFocus |  Spearman rank co-efficients
2019 |  [12] |  Seda Bilaloglu |  PathCNN |  AUC and $95\%$ confidence interval
2019 |  [79] |  Huangjing Lin |  Fast ScanNet (Based on FCN) |  FROC, AUC

##  6 Methodology Analysis

This section analyzes prominent methods in different tasks.

We can see from the above Table. 4 that FCN and U-net are mostly used in WSI
segmentation using ANN. From this, we can know that the model with skip
connection and encoder-decoder network like U-net is a successful model for
medical images. U-net is just the extension of FCN. FCN is that there is no
fully connected layer in the network, and it is replaced by the convolutional
layer. In addition, the skip connection [81] structure is used to refine the
segmentation. The schematic diagram of skip connection is shown in Figure. 33.
In this way, the network used for classification can be rewritten as the pixel
classification network used for segmentation. However, due to the lack of
consideration of context information between pixels, the segmentation result
is not precise enough. So some researchers use FCN and CRF [116] to further
optimize the segmentation results. U-net captures both the contracting path of
context information and the symmetric expanding path that allows accurate
positioning, which enables the network to propagate context information to a
higher resolution layer. Because the network can combine the low-resolution
information after downsampling with the high-resolution information after
upsampling, it is perfect for medical images with fuzzy boundaries and
accurate segmentation. However, the U-net structure can only be predicted on a
single scale, and can not cope with scale changes. Moreover, when the
convolution layer increases, the training does not have a good generalization
ability.

Figure 33: Jump connection structure of FCN. The figure matches to Figure.3 in
[81].

In the classification section, the number of cases is the largest. The
classification task is similar to segmentation, but the classification focuses
more on identifying and quantifying the focus area in the image.

From the Table. 5, we can see that using ANN to classify WSI, most of which
use CNN-based network architecture, which can make it possible to retain all
the information of the input image to the greatest extent. But there are few
papers that only use deep learning algorithms. Most of them are combined with
traditional machine learning algorithms, such as SVM, RF and so on [20, 127,
2, 135, 130]. Some use ANN for prediction and traditional machine learning
algorithms for classification [62]; some use ANN for pixel-level
classification and traditional machine learning classifiers for slide-level
classification [108].

The most noteworthy thing is that starting in 2018, there has been a task of
combining MIL and ANN to perform WSI classification. The concept of MIL is
first used by Dietterich et al. [32] in 1997 to solve the problem of unclear
training data. In MIL, the training set consists of a set of bags with
classification labels. Each bag contains several instances without
classification labels. If the bag contains at least one positive instance, the
bag is marked as a positive bag, and vice versa. Combining MIL and ANN for
classification can get better results. The CNN-based neural network is an end-
to-end architecture, similar to a black box, and is weak in interpretability
[140].

The detection is generally combined with classification. Generally, WSI is
tested first and then classified [10, 122, 79]. The detection task generally
targets the disease with breast cancer WSI [44, 22, 139, 122, 21, 78]. Its
network architecture is also based on the CNN model. The popular methods of
segmentation, classification, and detection for WSIs are as shown in Figure.
34.

Figure 34: The popular methods of segmentation, classification, and detection.

##  7 Potential Methods for WSI

###  7.1 Attention Mechanism

The Attention Model originally come from the study of human vision. Humans
selectively focus on some parts of all information while ignoring other
visible information. This mechanism is often referred to as the attentional
mechanism. It was first proposed for use in natural language processing [6].
Attention mechanism (AM) has become an important concept in the field of
neural networks. Its significance is to make the computer learn to pay
attention to the key information in the image and ignore the irrelevant
information [134]. Its principle is to pay attention to the input weight
allocation. It would be helpful if we could apply this mechanism in the field
of WSI classification, segmentation and detection to assist doctors in
diagnosis.

Nowadays, there are many applications of AM in the field of image processing.
In [61], a spatial transformer network (STN) is proposed. With this network,
the ROI can be located; the ROI can be transformed into an ideal image by an
affine transformation, then they are put into a neural network for training.
The STN module can be put into any region of CNN as a separate module, and the
input can be either the input image or the feature map. The network model used
is shown in Figure. 35.

Figure 35: The architecture of a spatial transformer module. This figure
matches to Figure.2 in [61].

In [61], AM is applied in the spatial domain, while in [57], the AM is applied
in the channel domain. The purpose of Squeeze-and-Excitation Networks (SENET)
is to learn the correlation between each channel, calculate the importance of
each channel, and get a set of weight values so that the next network can
selectively enhance the useful feature channel and suppress the useless
feature channel. The AM module is shown in the Figure. 36.

Figure 36: A Squeeze-and-Excitation block. The figure matches to Figure.1 in
[57].

In [132], AM is applied in both the spatial domain and channel domain. The
CBAM module proposed in this paper is divided into two parts: Channel
attention and spatial attention. Channel attention is the compression of
feature maps in the spatial dimension. Spatial attention is the compression of
feature maps in channel dimensions. This enables the network to clearly
understand which parts of the feature map should have a higher response at the
channel and spatial levels. Then the module is embedded into the neural
network structure for experiments.

###  7.2 Visual Transformer

The transformer is a DNN based on the self-attention mechanism, which is
initially applied in natural language processing. The emergence of the
transformer solves the large-scale machine translation tasks in natural
language processing. Because of the excellent performance of transformer in
natural language processing, the Visual Transformer (VT) is a technology-based
on the transformer in the field of computer vision [53]. The advantage of the
transformer is to capture the global information by using Multi-Head Self-
Attention(MHSA), to extract more powerful features. In the existing
transformer-based models, some are purely transformer structure models, others
are models using the combined CNN and transformer structure. Overall, VT can
better handle the classification, segmentation, and object detection tasks of
pathological WSIs.
In [34], the VT model is used to process an image classification task by
connecting the transformer encoder model and Multi-Layer Perceptron (MLP). It
is used to cut large-scale images into the patch and position encoding for
each patch to the transformer encoder model. Finally, the output features are
obtained through MLP to get the final classification results. Because
transformer models do not have the translation equivariance and locality of
CNNs, VT is not as effective in dealing with small datasets. However, using
transfer learning to pre-train VT on large-scale datasets greatly improves its
accuracy, reaching an accuracy of $88.36\%$ on ImageNet, $99.50\%$ on
CIFAR-10, $94.55\%$ on CIFAR-100, and $77.16\%$ on the VTAB suite of 19 tasks.

Figure 37: The VT architecture.

In [17], the DEtection TRansformer (DETR) model is an end-to-end object
detection method by adding position encodings before the transformer. It
abandons traditional hand-crafted components like anchor generation and Non-
maximum Suppression (NMS) post-processing. The result is equivalent to the
fast RCNN on the COCO dataset and easily migrates DETR to other tasks, such as
panoramic segmentation.

Figure 38: The DETR architecture.

In [128], inspired by DETR, the Max-DeepLab model directly predicts a set of
non-overlapping masks and their corresponding semantic tags through a mask
transformer for image segmentation. Moreover, unlike other models, which stack
transformer models directly on CNNs, the Max-DeepLab model adopts a dual-path
framework, which can best combine with the features extracted by CNN.

###  7.3 Other Potential Methods Applied to WSI Technology

In addition to AM and VT, there are some other potential methods that can be
applied to WSI to facilitate its segmentation, classification, and detection.

In [116], a method of gastric histopathological image segmentation based on
the layered CRF is introduced. This method can automatically locate the cancer
nest information in the stomach image, and because the CRF can represent the
spatial relationship, a higher-order term can be established on this basis and
applied to the image-based post-processing, which further improves the
segmentation performance. The model shows high subdivision performance and
effectiveness.

In [43], an image segmentation method is presented that is faster than
superpixel is proposed. It separates into dense and sparse methods. Then, a
new intensive method can achieve superior boundary adherence by exploring
alternative mid-level segmentation strategies are proposed. This method is a
very effective hierarchical segmentation method. But in this case, it applied
to natural images, and we can also try to apply it to WSI.

In [23], a method of automatic segmentation of coronary arteries based on a
growing algorithm is proposed. Firstly, 2D U-net is used to automatically
locate the initial seed points and the growth strategy, and then a growth
algorithm combined with the 3D triangulation network is proposed. The improved
3D U-net is used for coronary artery segmentation. This method adopts residual
block and two-phase training. The input data of the network is set as the
neighborhood block of the seed point. And according to the iterative
termination condition, it determines whether the segmentation is stopped.

In [88], a spectral space classification algorithm based on spectral space
feature fusion is proposed. The algorithm is used to classify hyperspectral
images by spatial coordinates. At the same time, active learning and the
combination of spectrum and space features are used to improve classification
performance and reduce the impact of noise.

In [29], two binary classifier systems are proposed, which can classify brain
magnetic resonance imaging (MRI) images. This method is novel and protects the
privacy of patients. Similarly, another paper on hyperspectral image
classification also deserves our attention.

In [96], the target in the video is detected. The method used is based on
traditional background subtraction and artificial intelligence detection. Mask
R-CNN is used to judge whether there is a segmentation object in the candidate
region and to segment it.

This paper [89] uses the traditional background subtraction and artificial
intelligence detection to detect the target in the video. We use the mask
R-CNN network architecture to judge whether there are segmented objects in the
candidate region, and then segment them.

In [86], a feature directed network is proposed by using Multi-scale Feature
Extraction Module (MFEM). The network can obtain the multi-scale context
information of each abstraction level for the complex scene of the target.
Moreover, a new loss function is designed, which is superior to the commonly
used cross-entropy loss function. This method does not need pretreatment and
can get better results. We can further apply it to histopathological WSI.

[52] uses two methods for target detection. The first method is to generate
semantic descriptors from a set of test scores of a single concept. Then the
semantic descriptor is sent to the multi-target concept detector as input. The
second method is used to detect multiple concepts and categories of objects.
Finally, the results of the two tests are summarized. From this paper, we can
see that the combination of context semantics and features based on deep
learning can produce good results.

##  8 Conclusions and Future Work

In this paper, a comprehensive overview of WSI analysis using ANN approaches
is given, including image segmentation, classification, and detection. Within
each of these tasks above, all related works are further reviewed according to
their detailed categories.

For the introduction of CAD using WSI and ANN, see Sect. 1. For the commonly
used ANN network architecture, see Sect. 2. For the commonly used WSI datasets
and evaluation indicators, see Sect. 3. In these commonly used datasets, TCGA
[4] and Camlyon [80] are the most used two datasets. The amount of work
related to WSI analysis using traditional ANNs are relatively small, and most
of them are analyzed using DNNs. The reason is that since the advent of WSI
technology in 1999, the large size of its image has been the reason why it is
difficult for researchers to make full use of it. It was not until 2006 that
related technologies began to be gradually developed and applied to pathology
[95]. At that time, the vigorous opportunity for the development of
traditional ANN has been missed. Therefore, DNNs are used to analyze
histopathological WSI. See Sect. 4, Sect. 5 for details. In addition to these
ANN methods, we have also proposed some possible WSI methods for
histopathology, such as AM and VT, see Sect. 7 for details. In short, as more
and more related technologies are proposed, they will be given to computers
auxiliary diagnosis brings better progress.

In the future, the application of neural networks in WSI CAD will further
improve the efficiency of pathologists, reduce the subjectivity of
pathologists’ diagnoses, and improve the safety of patients. Firstly, most of
the diseases analyzed by WSI are focused on the breast, gastric, colon, and
nervous systems. It is hoped that in the future, it can be extended to a wider
range and other diseases. Secondly, there is still a lack of comprehensive,
clear, fully annotated, and large-scale WSI datasets. Finally, we hope that it
will be very useful to develop a network with less computation and hardware
loss, higher efficiency, and interpretability.

## Acknowledgements

This work is supported by National Natural Science Foundation of China (No.
61806047). We thank Miss Zixian Li and Mr. Guoxian Li for their important
discussion. We also thank M.E. Xiaoming Zhou, M.E. Jinghua Zhang and B.E.
Jining Li, for their important technical supports.

## Conflict of Interest

The authors declare that they have no conflict of interest.

## References

  * AlZubaidi et al. [2017] AlZubaidi, A. K., Sideseq, F. B., Faeq, A., and Basil, M. (2017).  Computer aided diagnosis in digital pathology application: Review and perspective approach in lung cancer classification.  In 2017 annual conference on new trends in information & Communications technology applications (NTICT), pages 219–224. IEEE.
  * Araújo et al. [2017] Araújo, T., Aresta, G., Castro, E., Rouco, J., Aguiar, P., Eloy, C., Polónia, A., and Campilho, A. (2017).  Classification of breast cancer histology images using convolutional neural networks.  PloS one, 12(6):e0177544.
  * Arevalo et al. [2015] Arevalo, J., Cruz-Roa, A., Arias, V., Romero, E., and González, F. A. (2015).  An unsupervised feature learning framework for basal cell carcinoma image analysis.  Artificial intelligence in medicine, 64(2):131–145.
  * Atlas [2006] Atlas, T. C. G. (2006).  Website.  https://portal.gdc.cancer.gov/.
  * Babaie et al. [2017] Babaie, M., Kalra, S., Sriram, A., Mitcheltree, C., Zhu, S., Khatami, A., Rahnamayan, S., and Tizhoosh, H. R. (2017).  Classification and retrieval of digital pathology scans: A new dataset.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 8–16.
  * Bahdanau et al. [2014] Bahdanau, D., Cho, K., and Bengio, Y. (2014).  Neural machine translation by jointly learning to align and translate.  arXiv:1409.0473.
  * Bandi et al. [2018] Bandi, P., Geessink, O., Manson, Q., Van Dijk, M., Balkenhol, M., Hermsen, M., Bejnordi, B. E., Lee, B., Paeng, K., Zhong, A., et al. (2018).  From detection of individual metastases to classification of lymph node status at the patient level: the camelyon17 challenge.  IEEE transactions on medical imaging, 38(2):550–560.
  * Bándi et al. [2017] Bándi, P., van de Loo, R., Intezar, M., Geijs, D., Ciompi, F., van Ginneken, B., van der Laak, J., and Litjens, G. (2017).  Comparison of different methods for tissue segmentation in histopathological whole-slide images.  In 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), pages 591–595. IEEE.
  * Bejnordi et al. [2016] Bejnordi, B. E., Balkenhol, M., Litjens, G., Holland, R., Bult, P., Karssemeijer, N., and Van Der Laak, J. A. (2016).  Automated detection of dcis in whole-slide h&e stained breast histopathology images.  IEEE transactions on medical imaging, 35(9):2141–2150.
  * Bejnordi et al. [2017a] Bejnordi, B. E., Veta, M., Van Diest, P. J., Van Ginneken, B., Karssemeijer, N., Litjens, G., Van Der Laak, J. A., Hermsen, M., Manson, Q. F., Balkenhol, M., et al. (2017a).  Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer.  Jama, 318(22):2199–2210.
  * Bejnordi et al. [2017b] Bejnordi, B. E., Zuidhof, G., Balkenhol, M., Hermsen, M., Bult, P., van Ginneken, B., Karssemeijer, N., Litjens, G., and van der Laak, J. (2017b).  Context-aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images.  Journal of Medical Imaging, 4(4):044504.
  * Bilaloglu et al. [2019] Bilaloglu, S., Wu, J., Fierro, E., Sanchez, R. D., Ocampo, P. S., Razavian, N., Coudray, N., and Tsirigos, A. (2019).  Efficient pan-cancer whole-slide image classification and outlier detection using convolutional neural networks.  bioRxiv, page 633123.
  * Bishop [2006] Bishop, C. M. (2006).  Pattern recognition and machine learning.  springer.
  * Bouvrie [2006] Bouvrie, J. (2006).  Notes on convolutional neural networks.  In Pract.
  * Campanella et al. [2019] Campanella, G., Hanna, M. G., Geneslaw, L., Miraflor, A., Silva, V. W. K., Busam, K. J., Brogi, E., Reuter, V. E., Klimstra, D. S., and Fuchs, T. J. (2019).  Clinical-grade computational pathology using weakly supervised deep learning on whole slide images.  Nature medicine, 25(8):1301–1309.
  * Campanella et al. [2018] Campanella, G., Silva, V. W. K., and Fuchs, T. J. (2018).  Terabyte-scale deep multiple instance learning for classification and localization in pathology.  arXiv:1805.06983.
  * Carion et al. [2020] Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020).  End-to-end object detection with transformers.  In European Conference on Computer Vision, pages 213–229. Springer.
  * Cortes and Vapnik [1995] Cortes, C. and Vapnik, V. (1995).  Support-vector networks.  Machine learning, 20(3):273–297.
  * Courtiol et al. [2018] Courtiol, P., Tramel, E. W., Sanselme, M., and Wainrib, G. (2018).  Classification and disease localization in histopathology using only global labels: A weakly-supervised approach.  arXiv:1802.02212.
  * Cruz-Roa et al. [2014] Cruz-Roa, A., Basavanhally, A., González, F., Gilmore, H., Feldman, M., Ganesan, S., Shih, N., Tomaszewski, J., and Madabhushi, A. (2014).  Automatic detection of invasive ductal carcinoma in whole slide images with convolutional neural networks.  In Medical Imaging 2014: Digital Pathology, volume 9041, page 904103\. International Society for Optics and Photonics.
  * Cruz-Roa et al. [2018] Cruz-Roa, A., Gilmore, H., Basavanhally, A., Feldman, M., Ganesan, S., Shih, N., Tomaszewski, J., Madabhushi, A., and González, F. (2018).  High-throughput adaptive sampling for whole-slide histopathology image analysis (hashi) via convolutional neural networks: Application to invasive breast cancer detection.  PloS one, 13(5):e0196828.
  * Cruz-Roa et al. [2017] Cruz-Roa, A., Gilmore, H., Basavanhally, A., Feldman, M., Ganesan, S., Shih, N. N., Tomaszewski, J., González, F. A., and Madabhushi, A. (2017).  Accurate and reproducible invasive breast cancer detection in whole-slide images: A deep learning approach for quantifying tumor extent.  Scientific reports, 7:46450.
  * Cui et al. [2020] Cui, J., Guo, H., Wang, H., Chen, F., Shu, L., and Li, L. C. (2020).  Fully-automatic segmentation of coronary artery using growing algorithm.  Journal of X-Ray Science and Technology, 55(Preprint):1–16.
  * Cui et al. [2019] Cui, Y., Zhang, G., Liu, Z., Xiong, Z., and Hu, J. (2019).  A deep learning algorithm for one-step contour aware nuclei segmentation of histopathology images.  Medical & biological engineering & computing, 57(9):2027–2043.
  * Cutler and Zhao [2001] Cutler, A. and Zhao, G. (2001).  Pert-perfect random tree ensembles.  Computing Science and Statistics, 33:490–497.
  * Das et al. [2018] Das, K., Conjeti, S., Roy, A. G., Chatterjee, J., and Sheet, D. (2018).  Multiple instance learning of deep convolutional neural networks for breast histopathology whole slide classification.  In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pages 578–581. IEEE.
  * Das et al. [2017] Das, K., Karri, S. P. K., Roy, A. G., Chatterjee, J., and Sheet, D. (2017).  Classifying histopathology whole-slides using fusion of decisions from deep convolutional network on a collection of random multi-views at multi-magnification.  In 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), pages 1024–1027. IEEE.
  * Della Mea et al. [2006] Della Mea, V., Demichelis, F., Viel, F., Dalla Palma, P., and Beltrami, C. A. (2006).  User attitudes in analyzing digital slides in a quality control test bed: A preliminary study.  Computer methods and programs in biomedicine, 82(2):177–186.
  * Devi et al. [2020] Devi, S., Sahoo, M. N., and Bakshi, S. (2020).  A novel privacy-supporting 2-class classification technique for brain mri images.  Biocybernetics and Biomedical Engineering.
  * Dharwal and Kaur [2016] Dharwal, R. and Kaur, L. (2016).  Applications of artificial neural networks: a review.  Indian J. Sci. Technol, 9(47):1–8.
  * Dhillon and Verma [2020] Dhillon, A. and Verma, G. K. (2020).  Convolutional neural network: a review of models, methodologies and applications to object detection.  Progress in Artificial Intelligence, 9(2):85–112.
  * Dietterich et al. [1997] Dietterich, T. G., Lathrop, R. H., and Lozano-Pérez, T. (1997).  Solving the multiple instance problem with axis-parallel rectangles.  Artificial intelligence, 89(1-2):31–71.
  * Dong et al. [2018] Dong, N., Kampffmeyer, M., Liang, X., Wang, Z., Dai, W., and Xing, E. (2018).  Reinforced auto-zoom net: Towards accurate and fast breast cancer segmentation in whole-slide images.  In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, pages 317–325. Springer.
  * Dosovitskiy et al. [2020] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020).  An image is worth 16x16 words: Transformers for image recognition at scale.  arXiv:2010.11929.
  * Doyle et al. [2010] Doyle, S., Feldman, M., Tomaszewski, J., and Madabhushi, A. (2010).  A boosted bayesian multiresolution classifier for prostate cancer detection from digitized needle biopsies.  IEEE transactions on biomedical engineering, 59(5):1205–1218.
  * Durand et al. [2016] Durand, T., Thome, N., and Cord, M. (2016).  Weldon: Weakly supervised learning of deep convolutional neural networks.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4743–4752.
  * Egan et al. [1961] Egan, J. P., Greenberg, G. Z., and Schulman, A. I. (1961).  Operating characteristics, signal detectability, and the method of free response.  The Journal of the Acoustical Society of America, 33(8):993–1007.
  * Evered and Dudding [2011] Evered, A. and Dudding, N. (2011).  Accuracy and perceptions of virtual microscopy compared with glass slide microscopy in cervical cytology.  Cytopathology, 22(2):82–87.
  * Farahani et al. [2015] Farahani, N., Parwani, A. V., and Pantanowitz, L. (2015).  Whole slide imaging in pathology: advantages, limitations, and emerging perspectives.  Pathology and Laboratory Medicine International, 7:23–33.
  * Feng et al. [2020] Feng, Y., Hafiane, A., and Laurent, H. (2020).  A deep learning based multiscale approach to segment cancer area in liver whole slide image.  arXiv:2007.12935.
  * Fukushima and Miyake [1982] Fukushima, K. and Miyake, S. (1982).  Neocognitron: A self-organizing neural network model for a mechanism of visual pattern recognition.  In Competition and cooperation in neural nets, pages 267–285. Springer.
  * Fukushima et al. [1983] Fukushima, K., Miyake, S., and Ito, T. (1983).  Neocognitron: A neural network model for a mechanism of visual pattern recognition.  IEEE transactions on systems, man, and cybernetics, SMC-13(5):826–834.
  * Galvão et al. [2020] Galvão, F. L., Guimarães, S. J. F., and Falcão, A. X. (2020).  Image segmentation using dense and sparse hierarchies of superpixels.  Pattern Recognition, 108:107532.
  * Geçer [2016] Geçer, B. (2016).  Detection and classification of breast cancer in whole slide histopathology images using deep convolutional networks.  Diss. Bilkent University, 1.
  * Gecer et al. [2018] Gecer, B., Aksoy, S., Mercan, E., Shapiro, L. G., Weaver, D. L., and Elmore, J. G. (2018).  Detection and classification of cancer in whole slide breast histopathology images using deep convolutional networks.  Pattern recognition, 84:345–356.
  * Ghosh et al. [2017] Ghosh, A., Singh, S., and Sheet, D. (2017).  Simultaneous localization and classification of acute lymphoblastic leukemic cells in peripheral blood smears using a deep convolutional network with average pooling layer.  In 2017 IEEE International Conference on Industrial and Information Systems (ICIIS), pages 1–6. IEEE.
  * Glatz-Krieger et al. [2006] Glatz-Krieger, K., Spornitz, U., Spatz, A., Mihatsch, M. J., and Glatz, D. (2006).  Factors to keep in mind when introducing virtual microscopy.  Virchows Archiv, 448(3):248–255.
  * Goacher et al. [2017] Goacher, E., Randell, R., Williams, B., and Treanor, D. (2017).  The diagnostic concordance of whole slide imaging and light microscopy: a systematic review.  Archives of pathology & laboratory medicine, 141(1):151–161.
  * Goodfellow et al. [2016] Goodfellow, I., Bengio, Y., Courville, A., and Bengio, Y. (2016).  Deep learning (vol. 1, no. 2).
  * Gu et al. [2018] Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., Cai, J., et al. (2018).  Recent advances in convolutional neural networks.  Pattern Recognition, 77:354–377.
  * Gutman et al. [2013] Gutman, D. A., Cobb, J., Somanna, D., Park, Y., Wang, F., Kurc, T., Saltz, J. H., Brat, D. J., Cooper, L. A., and Kong, J. (2013).  Cancer digital slide archive: an informatics resource to support integrated in silico analysis of tcga pathology data.  Journal of the American Medical Informatics Association, 20(6):1091–1098.
  * Hamadi et al. [2020] Hamadi, A., Lattar, H., Khoussa, M. E. B., and Safadi, B. (2020).  Using semantic context for multiple concepts detection in still images.  Pattern Analysis and Applications, 23(1):27–44.
  * Han et al. [2020] Han, K., Wang, Y., Chen, H., Chen, X., Guo, J., Liu, Z., Tang, Y., Xiao, A., Xu, C., Xu, Y., et al. (2020).  A survey on visual transformer.  arXiv:2012.12556.
  * Haralick and Shapiro [1985] Haralick, R. M. and Shapiro, L. G. (1985).  Image segmentation techniques.  Computer vision, graphics, and image processing, 29(1):100–132.
  * He et al. [2016] He, K., Zhang, X., Ren, S., and Sun, J. (2016).  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778.
  * Hou et al. [2016] Hou, L., Samaras, D., Kurc, T. M., Gao, Y., Davis, J. E., and Saltz, J. H. (2016).  Patch-based convolutional neural network for whole slide tissue image classification.  In Proceedings of the ieee conference on computer vision and pattern recognition, pages 2424–2433.
  * Hu et al. [2018] Hu, J., Shen, L., and Sun, G. (2018).  Squeeze-and-excitation networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7132–7141.
  * Hu et al. [2017] Hu, J.-X., Xu, Y.-Y., Shen, H.-B., et al. (2017).  Deep learning-based classification of protein subcellular localization from immunohistochemistry images.  In 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR), pages 599–604. IEEE.
  * Hubel and Wiesel [1968] Hubel, D. H. and Wiesel, T. N. (1968).  Receptive fields and functional architecture of monkey striate cortex.  The Journal of physiology, 195(1):215–243.
  * Ioffe and Szegedy [2015] Ioffe, S. and Szegedy, C. (2015).  Batch normalization: Accelerating deep network training by reducing internal covariate shift.  In International conference on machine learning, pages 448–456. PMLR.
  * Jaderberg et al. [2015] Jaderberg, M., Simonyan, K., Zisserman, A., and Kavukcuoglu, K. (2015).  Spatial transformer networks.  arXiv:1506.02025.
  * Jamaluddin et al. [2017] Jamaluddin, M. F., Fauzi, M. F. A., and Abas, F. S. (2017).  Tumor detection and whole slide classification of h&e lymph node images using convolutional neural network.  In 2017 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), pages 90–95. IEEE.
  * Jiang et al. [2019] Jiang, H., Piao, S., Qadir, M. Z., and Guo, Q. (2019).  M-wrsf model for medical image segmentation.  Electronics Letters, 55(7):386–389.
  * Jiao et al. [2013] Jiao, L., Chen, Q., Li, S., and Xu, Y. (2013).  Colon cancer detection using whole slide histopathological images.  In World Congress on Medical Physics and Biomedical Engineering May 26-31, 2012, Beijing, China, pages 1283–1286. Springer.
  * Kalogirou [2001] Kalogirou, S. A. (2001).  Artificial neural networks in renewable energy systems applications: a review.  Renewable and sustainable energy reviews, 5(4):373–401.
  * Kamavisdar et al. [2013] Kamavisdar, P., Saluja, S., and Agrawal, S. (2013).  A survey on image classification approaches and techniques.  International Journal of Advanced Research in Computer and Communication Engineering, 2(1):1005–1009.
  * Kohlberger et al. [2019] Kohlberger, T., Liu, Y., Moran, M., Chen, P.-H. C., Brown, T., Hipp, J. D., Mermel, C. H., and Stumpe, M. C. (2019).  Whole-slide image focus quality: Automatic assessment and impact on ai cancer detection.  Journal of Pathology Informatics, 10.
  * Komura and Ishikawa [2018] Komura, D. and Ishikawa, S. (2018).  Machine learning methods for histopathological image analysis.  Computational and structural biotechnology journal, 16:34–42.
  * Korbar et al. [2017a] Korbar, B., Olofson, A. M., Miraflor, A. P., Nicka, C. M., Suriawinata, M. A., Torresani, L., Suriawinata, A. A., and Hassanpour, S. (2017a).  Deep learning for classification of colorectal polyps on whole-slide images.  Journal of pathology informatics, 8.
  * Korbar et al. [2017b] Korbar, B., Olofson, A. M., Miraflor, A. P., Nicka, C. M., Suriawinata, M. A., Torresani, L., Suriawinata, A. A., and Hassanpour, S. (2017b).  Looking under the hood: Deep neural network visualization to interpret whole-slide image analysis outcomes for colorectal polyps.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 69–75.
  * Krizhevsky et al. [2012] Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2012).  Imagenet classification with deep convolutional neural networks.  Advances in neural information processing systems, 25:1097–1105.
  * Kumar et al. [2018] Kumar, M. D., Babaie, M., and Tizhoosh, H. R. (2018).  Deep barcodes for fast retrieval of histopathology scans.  In 2018 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE.
  * Kumar et al. [2005] Kumar, V., Abbas, A., and Fausto, N. (2005).  Robbins and cotran pathologic basis of disease. elsevier saunders.  Philadelphia, PA [Google Scholar].
  * Kwok [2018] Kwok, S. (2018).  Multiclass classification of breast cancer in whole-slide images.  In International conference image analysis and recognition, pages 931–940. Springer.
  * Le Cun et al. [1989] Le Cun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., and Jackel, L. D. (1989).  Handwritten digit recognition with a back-propagation network.  In Proceedings of the 2nd International Conference on Neural Information Processing Systems, pages 396–404.
  * Li et al. [2019a] Li, C., Wang, K., and Xu, N. (2019a).  A survey for the applications of content-based microscopic image analysis in microorganism classification domains.  Artificial Intelligence Review, 51(4):577–646.
  * Li et al. [2019b] Li, J., Li, W., Gertych, A., Knudsen, B. S., Speier, W., and Arnold, C. W. (2019b).  An attention-based multi-resolution model for prostate whole slide imageclassification and localization.  arXiv:1905.13208.
  * Lin et al. [2018] Lin, H., Chen, H., Dou, Q., Wang, L., Qin, J., and Heng, P.-A. (2018).  Scannet: A fast and dense scanning framework for metastastic breast cancer detection from whole-slide image.  In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 539–546. IEEE.
  * Lin et al. [2019] Lin, H., Chen, H., Graham, S., Dou, Q., Rajpoot, N., and Heng, P.-A. (2019).  Fast scannet: Fast and dense analysis of multi-gigapixel whole-slide images for cancer metastasis detection.  IEEE transactions on medical imaging, 38(8):1948–1958.
  * Litjens et al. [2018] Litjens, G., Bandi, P., Ehteshami Bejnordi, B., Geessink, O., Balkenhol, M., Bult, P., Halilovic, A., Hermsen, M., van de Loo, R., Vogels, R., et al. (2018).  1399 h&e-stained sentinel lymph node sections of breast cancer patients: the camelyon dataset.  GigaScience, 7(6):giy065.
  * Long et al. [2015] Long, J., Shelhamer, E., and Darrell, T. (2015).  Fully convolutional networks for semantic segmentation.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3431–3440.
  * Maksoud et al. [2019] Maksoud, S., Wiliem, A., Zhao, K., Zhang, T., Wu, L., and Lovell, B. (2019).  Coral8: Concurrent object regression for area localization in medical image panels.  In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 432–441. Springer.
  * Manyala et al. [2019] Manyala, A., Cholakkal, H., Anand, V., Kanhangad, V., and Rajan, D. (2019).  Cnn-based gender classification in near-infrared periocular images.  Pattern Analysis and Applications, 22(4):1493–1504.
  * McCulloch and Pitts [1943] McCulloch, W. S. and Pitts, W. (1943).  A logical calculus of the ideas immanent in nervous activity.  The bulletin of mathematical biophysics, 5(4):115–133.
  * Mehta et al. [2018] Mehta, S., Mercan, E., Bartlett, J., Weaver, D., Elmore, J., and Shapiro, L. (2018).  Learning to segment breast biopsy whole slide images.  In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 663–672. IEEE.
  * Mohammadi et al. [2020] Mohammadi, S., Noori, M., Bahri, A., Majelan, S. G., and Havaei, M. (2020).  Cagnet: Content-aware guidance for salient object detection.  Pattern Recognition, page 107303.
  * Mohri et al. [2018] Mohri, M., Rostamizadeh, A., and Talwalkar, A. (2018).  Foundations of machine learning.  MIT press.
  * Mu et al. [2020] Mu, C., Liu, J., Liu, Y., and Liu, Y. (2020).  Hyperspectral image classification based on active learning and spectral-spatial feature fusion using spatial coordinates.  IEEE Access, 8:6768–6781.
  * Mukherjee et al. [2020] Mukherjee, J., Kar, M., Chakrabarti, A., and Das, S. (2020).  A soft-computing based approach towards automatic detection of pulmonary nodule.  Biocybernetics and Biomedical Engineering.
  * Nayak et al. [2013] Nayak, N., Chang, H., Borowsky, A., Spellman, P., and Parvin, B. (2013).  Classification of tumor histopathology via sparse feature learning.  In 2013 IEEE 10th international symposium on biomedical imaging, pages 410–413. IEEE.
  * Nirschl et al. [2018] Nirschl, J. J., Janowczyk, A., Peyster, E. G., Frank, R., Margulies, K. B., Feldman, M. D., and Madabhushi, A. (2018).  A deep-learning classifier identifies patients with clinical heart failure using whole-slide images of h&e tissue.  PloS one, 13(4):e0192726.
  * Orlov et al. [2008] Orlov, N., Shamir, L., Macura, T., Johnston, J., Eckley, D. M., and Goldberg, I. G. (2008).  Wnd-charm: Multi-purpose image classification using compound image transforms.  Pattern recognition letters, 29(11):1684–1693.
  * O’Shea and Nash [2015] O’Shea, K. and Nash, R. (2015).  An introduction to convolutional neural networks.  arXiv:1511.08458.
  * Pal and Pal [1993] Pal, N. R. and Pal, S. K. (1993).  A review on image segmentation techniques.  Pattern recognition, 26(9):1277–1294.
  * Pantanowitz et al. [2011] Pantanowitz, L., Valenstein, P. N., Evans, A. J., Kaplan, K. J., Pfeifer, J. D., Wilbur, D. C., Collins, L. C., and Colgan, T. J. (2011).  Review of the current state of whole slide imaging in pathology.  Journal of pathology informatics, 2.
  * Park et al. [2020] Park, H., Park, S., and Joo, Y. (2020).  Detection of abandoned and stolen objects based on dual background model and mask r-cnn.  IEEE Access, 8:80010–80019.
  * Petushi et al. [2006] Petushi, S., Garcia, F. U., Haber, M. M., Katsinis, C., and Tozeren, A. (2006).  Large-scale computations on histology images reveal grade-differentiating parameters for breast cancer.  BMC medical imaging, 6(1):14.
  * Ren et al. [2018a] Ren, J., Hacihaliloglu, I., Singer, E. A., Foran, D. J., and Qi, X. (2018a).  Adversarial domain adaptation for classification of prostate histopathology whole-slide images.  In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 201–209. Springer.
  * Ren et al. [2018b] Ren, J., Karagoz, K., Gatza, M., Foran, D. J., and Qi, X. (2018b).  Differentiation among prostate cancer patients with gleason score of 7 using histopathology whole-slide image and genomic data.  In Medical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications, volume 10579, page 1057904. International Society for Optics and Photonics.
  * Romer et al. [2003] Romer, D. J., Yearsley, K. H., and Ayers, L. W. (2003).  Using a modified standard microscope to generate virtual slides.  The Anatomical Record Part B: The New Anatomist: An Official Publication of the American Association of Anatomists, 272(1):91–97.
  * Schrader et al. [2006] Schrader, T., Niepage, S., Leuthold, T., Saeger, K., Schluns, K., Hufnagl, P., Kayser, K., and Dietel, M. (2006).  The diagnostic path, a useful visualisation tool in virtual microscopy.  Diagnostic Pathology, 1(1):1–7.
  * Seth [2019] Seth, N. (2019).  Automated Localization of Breast Ductal Carcinoma in Situ in Whole Slide Images.  PhD thesis, University of Toronto (Canada).
  * Seth et al. [2019] Seth, N., Akbar, S., Nofech-Mozes, S., Salama, S., and Martel, A. L. (2019).  Automated segmentation of dcis in whole slide images.  In European Congress on Digital Pathology, pages 67–74. Springer.
  * Sharma et al. [2016] Sharma, H., Zerbe, N., Klempert, I., Hellwich, O., and Hufnagl, P. (2016).  Deep convolutional neural networks for histological image analysis in gastric carcinoma whole slide images.  Diagnostic Pathology, 1(8).
  * Sharma et al. [2017] Sharma, H., Zerbe, N., Klempert, I., Hellwich, O., and Hufnagl, P. (2017).  Deep convolutional neural networks for automatic classification of gastric carcinoma using whole slide images in digital histopathology.  Computerized Medical Imaging and Graphics, 61:2–13.
  * Sharma et al. [2012] Sharma, V., Rai, S., and Dev, A. (2012).  A comprehensive study of artificial neural networks.  International Journal of Advanced research in computer science and software engineering, 2(10).
  * Sheikhzadeh et al. [2016] Sheikhzadeh, F., Guillaud, M., and Ward, R. K. (2016).  Automatic labeling of molecular biomarkers of whole slide immunohistochemistry images using fully convolutional networks.  arXiv:1612.09420.
  * Shou et al. [2018] Shou, J., Li, Y., Yu, G., and Li, G. (2018).  Whole slide image classification of gastric cancer using convolutional neural networks.  OpenReview : MIDL 2018.
  * Simonyan and Zisserman [2014] Simonyan, K. and Zisserman, A. (2014).  Very deep convolutional networks for large-scale image recognition.  arXiv:1409.1556.
  * Sirinukunwattana et al. [2018] Sirinukunwattana, K., Alham, N. K., Verrill, C., and Rittscher, J. (2018).  Improving whole slide segmentation through visual context-a systematic study.  In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 192–200. Springer.
  * Sirinukunwattana et al. [2016] Sirinukunwattana, K., Raza, S. E. A., Tsang, Y.-W., Snead, D. R., Cree, I. A., and Rajpoot, N. M. (2016).  Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images.  IEEE transactions on medical imaging, 35(5):1196–1206.
  * Sokolova and Lapalme [2009] Sokolova, M. and Lapalme, G. (2009).  A systematic analysis of performance measures for classification tasks.  Information processing & management, 45(4):427–437.
  * Sornapudi et al. [2020a] Sornapudi, S., Addanki, R., Stanley, J., Stoecker, W. V., Long, R., Zuna, R., Frazier, S. R., and Antani, S. (2020a).  Cervical whole slide histology image analysis toolbox.  medRxiv.
  * Sornapudi et al. [2020b] Sornapudi, S., Hagerty, J., Stanley, R. J., Stoecker, W. V., Long, R., Antani, S., Thoma, G., Zuna, R., and Frazier, S. R. (2020b).  Epithnet: Deep regression for epithelium segmentation in cervical histology images.  Journal of Pathology Informatics, 11.
  * Sornapudi et al. [2020c] Sornapudi, S., Stanley, R. J., Stoecker, W. V., Long, R., Xue, Z., Zuna, R., Frazier, S. R., and Antani, S. (2020c).  Feature based sequential classifier with attention mechanism.  arXiv:2007.11392.
  * Sun et al. [2020] Sun, C., Li, C., Zhang, J., Rahaman, M. M., Ai, S., Chen, H., Kulwa, F., Li, Y., Li, X., and Jiang, T. (2020).  Gastric histopathology image segmentation using a hierarchical conditional random field.  Biocybernetics and Biomedical Engineering, 40(4):1535–1555.
  * Suzuki [2011] Suzuki, K. (2011).  Artificial neural networks: methodological advances and biomedical applications.  BoD–Books on Demand.
  * Swiderska et al. [2015] Swiderska, Z., Markiewicz, T., Grala, B., and Kozlowski, W. (2015).  Texture and mathematical morphology for hot-spot detection in whole slide images of meningiomas and oligodendrogliomas.  In International Conference on Computer Analysis of Images and Patterns, pages 1–12. Springer.
  * Szegedy et al. [2016a] Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A. (2016a).  Inception-resnet and the impact of residual connections on learning.  arXiv:1602.07261.
  * Szegedy et al. [2015] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015).  Going deeper with convolutions.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1–9.
  * Szegedy et al. [2016b] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016b).  Rethinking the inception architecture for computer vision.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818–2826.
  * Tellez et al. [2018a] Tellez, D., Balkenhol, M., Otte-Höller, I., van de Loo, R., Vogels, R., Bult, P., Wauters, C., Vreuls, W., Mol, S., Karssemeijer, N., et al. (2018a).  Whole-slide mitosis detection in h&e breast histology using phh3 as a reference to train distilled stain-invariant convolutional networks.  IEEE transactions on medical imaging, 37(9):2126–2136.
  * Tellez et al. [2018b] Tellez, D., van der Laak, J., and Ciompi, F. (2018b).  Gigapixel whole-slide image classification using unsupervised image compression and contrastive training.  Med. Imag. Deep Learn.
  * Tomczak et al. [2015] Tomczak, K., Czerwińska, P., and Wiznerowicz, M. (2015).  The cancer genome atlas (tcga): an immeasurable source of knowledge.  Contemporary oncology, 19(1A):A68.
  * Veta et al. [2019] Veta, M., Heng, Y. J., Stathonikos, N., Bejnordi, B. E., Beca, F., Wollmann, T., Rohr, K., Shah, M. A., Wang, D., Rousson, M., et al. (2019).  Predicting breast tumor proliferation from whole-slide images: the tupac16 challenge.  Medical image analysis, 54:111–121.
  * Veta et al. [2015] Veta, M., Van Diest, P. J., Willems, S. M., Wang, H., Madabhushi, A., Cruz-Roa, A., Gonzalez, F., Larsen, A. B., Vestergaard, J. S., Dahl, A. B., et al. (2015).  Assessment of algorithms for mitosis detection in breast cancer histopathology images.  Medical image analysis, 20(1):237–248.
  * Wang et al. [2016] Wang, D., Khosla, A., Gargeya, R., Irshad, H., and Beck, A. H. (2016).  Deep learning for identifying metastatic breast cancer.  arXiv:1606.05718.
  * Wang et al. [2020] Wang, H., Zhu, Y., Adam, H., Yuille, A., and Chen, L.-C. (2020).  Max-deeplab: End-to-end panoptic segmentation with mask transformers.  arXiv:2012.00759.
  * Wang et al. [2019a] Wang, S., Zhu, Y., Yu, L., Chen, H., Lin, H., Wan, X., Fan, X., and Heng, P.-A. (2019a).  Rmdl: Recalibrated multi-instance deep learning for whole slide gastric image classification.  Medical image analysis, 58:101549.
  * Wang et al. [2018] Wang, X., Chen, H., Gan, C., Lin, H., Dou, Q., Huang, Q., Cai, M., and Heng, P.-A. (2018).  Weakly supervised learning for whole slide lung cancer image classification.  Med. Imag. Deep Learn.
  * Wang et al. [2019b] Wang, X., Chen, H., Gan, C., Lin, H., Dou, Q., Tsougenis, E., Huang, Q., Cai, M., and Heng, P.-A. (2019b).  Weakly supervised deep learning for whole slide lung cancer image analysis.  IEEE transactions on cybernetics, 50(9):3950–3962.
  * Woo et al. [2018] Woo, S., Park, J., Lee, J.-Y., and Kweon, I. S. (2018).  Cbam: Convolutional block attention module.  In Proceedings of the European conference on computer vision (ECCV), pages 3–19.
  * Xu et al. [2019] Xu, J., Cai, C., Zhou, Y., Yao, B., Xu, G., Wang, X., Zhao, K., Madabhushi, A., Liu, Z., and Liang, L. (2019).  Multi-tissue partitioning for whole slide images of colorectal cancer histopathology images with deeptissue net.  In European Congress on Digital Pathology, pages 100–108. Springer.
  * Xu et al. [2015] Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., Zemel, R., and Bengio, Y. (2015).  Show, attend and tell: Neural image caption generation with visual attention.  In International conference on machine learning, pages 2048–2057. PMLR.
  * Xu et al. [2017] Xu, Y., Jia, Z., Wang, L.-B., Ai, Y., Zhang, F., Lai, M., Eric, I., and Chang, C. (2017).  Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features.  BMC bioinformatics, 18(1):1–17.
  * Yamashita et al. [2018] Yamashita, R., Nishio, M., Do, R. K. G., and Togashi, K. (2018).  Convolutional neural networks: an overview and application in radiology.  Insights into imaging, 9(4):611–629.
  * Yoshida et al. [2018] Yoshida, H., Shimazu, T., Kiyuna, T., Marugame, A., Yamashita, Y., Cosatto, E., Taniguchi, H., Sekine, S., and Ochiai, A. (2018).  Automated histological classification of whole-slide images of gastric biopsy specimens.  Gastric Cancer, 21(2):249–257.
  * Yue et al. [2019] Yue, X., Dimitriou, N., and Arandjelovic, O. (2019).  Colorectal cancer outcome prediction from h&e whole slide images using machine learning and automatically inferred phenotype profiles.  arXiv:1902.03582.
  * Zanjani et al. [2018] Zanjani, F. G., Zinger, S., et al. (2018).  Cancer detection in histopathology whole-slide images using conditional random fields on deep embedded spaces.  In Medical imaging 2018: Digital pathology, volume 10581, page 105810I. International Society for Optics and Photonics.
  * ZHANG [2013] ZHANG, C. (2013).  Challenges in machine learning.  SCIENTIA SINICA Informationis, 43(12):1612–1623.
  * Zhou et al. [2020] Zhou, X., Li, C., Rahaman, M. M., Yao, Y., Ai, S., Sun, C., Wang, Q., Zhang, Y., Li, M., Li, X., et al. (2020).  A comprehensive review for breast histopathology image analysis using classical and deep neural networks.  IEEE Access, 8:90931–90956.