# Artificial Intelligence in Ovarian Cancer Histopathology: A Systematic
Review

Jack Breen222Corresponding author -<EMAIL_ADDRESS>Centre for
Computational Imaging and Simulation Technologies in Biomedicine (CISTIB),
School of Computing, University of Leeds, UK  Katie Allen  Leeds Institute of
Medical Research at St James’s, School of Medicine, University of Leeds, UK
Kieran Zucker  Leeds Cancer Centre, St James’s University Hospital, Leeds, UK
Pratik Adusumilli  Leeds Institute of Medical Research at St James’s, School
of Medicine, University of Leeds, UK  Department of Radiology, St. James
University Hospital, Leeds, UK  Andy Scarsbrook  Leeds Institute of Medical
Research at St James’s, School of Medicine, University of Leeds, UK
Department of Radiology, St. James University Hospital, Leeds, UK  Geoff Hall
Leeds Cancer Centre, St James’s University Hospital, Leeds, UK  Nicolas M.
Orsi  Leeds Institute of Medical Research at St James’s, School of Medicine,
University of Leeds, UK  Nishant Ravikumar  Centre for Computational Imaging
and Simulation Technologies in Biomedicine (CISTIB), School of Computing,
University of Leeds, UK

###### Abstract

### Purpose

To characterise and assess the quality of published research evaluating
artificial intelligence (AI) methods for ovarian cancer diagnosis or prognosis
using histopathology data.

### Methods

A search of PubMed, Scopus, Web of Science, Cochrane Central Register of
Controlled Trials, and WHO International Clinical Trials Registry Platform was
conducted up to 01/12/2022. The inclusion criteria required that research
evaluated AI on histopathology images for diagnostic or prognostic inferences
in ovarian cancer, including primary tumours of the ovaries, fallopian tubes,
and peritoneum. Reviews and non-English language articles were excluded. The
risk of bias was assessed for every model that met the inclusion criteria
using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). Information
about each model of interest was tabulated and summary statistics were
reported. Based on the results, we provided recommendations to improve study
design and reporting to reduce the risk of bias and improve the
reproducibility of future research in the field. The study protocol was
registered on PROSPERO (CRD42022334730). PRISMA 2020 reporting guidelines were
followed.

### Results

A total of 1434 research articles were identified, of which 36 were eligible
for inclusion. These studies reported 62 models of interest, including 35
classifiers, 14 survival prediction models, 7 segmentation models, and 6
regression models. Models were developed using 1-1375 slides from 1-664
ovarian cancer patients. A wide array of outcomes were predicted, including
overall survival (9/62), histological subtypes (7/62), stain quantity (6/62),
malignancy (5/62), primary cancer (4/62), and tumour region (4/62). Older
studies used traditional machine learning (ML) models with hand-crafted
features, while newer studies typically employed deep learning (DL) to
automatically learn features and predict the outcome(s) of interest. All
models were found to be at high or unclear risk of bias overall, with most
research having a high risk of bias in the analysis and a lack of clarity
regarding participants and predictors in the study. Research was frequently
limited by insufficient reporting, small sample sizes, and insufficient
validation, with external validation being particularly rare.

### Conclusion

Limited research has been conducted on the application of AI to histopathology
images for diagnostic or prognostic purposes in ovarian cancer, and none of
the associated models have been demonstrated to be ready for real-world
implementation. Recommendations are provided addressing underlying biases and
flaws in study design, which should help inform higher-quality reproducible
future research. Key aspects to help ensure clinical translation include more
transparent and comprehensive reporting of data provenance and modelling
approaches, as well as improved quantitative performance evaluation using
cross-validation and external validations.

### Funding

UKRI Engineering and Physical Sciences Research Council and The Tony Bramall
Charitable Trust

## Introduction

Ovarian cancer is the eighth most common malignancy in women worldwide [1]. It
is notoriously difficult to detect and diagnose, with ineffective screening
[2] and vague symptoms similar to those caused by menopause [3]. Encompassing
primary malignant tumours of the ovaries, fallopian tubes, and peritoneum, the
disease has often started to spread within the abdomen at the time of
diagnosis (FIGO [4] Stage 3). This typical late stage at diagnosis makes
ovarian cancer a particularly deadly disease, with the 314,000 new cases
diagnosed each year translating to 207,000 deaths a year globally [1].

Most ovarian cancers are carcinomas (cancers of epithelial origin) which
predominantly fall into five histological subtypes: high-grade serous, low-
grade serous, clear cell, endometrioid, and mucinous. Non-epithelial ovarian
cancers are rare and include germ cell, sex cord-stromal, and mesenchymal
tumours. Ovarian cancer subtypes differ morphologically and prognostically and
have varying treatment options [5]. High-grade serous carcinoma is by far the
most common form of ovarian cancer, accounting for approximately 70% of all
cases [6].

Histopathology, the examination of tissue specimens at the cellular level, is
the gold standard for ovarian cancer diagnosis. Pathologists typically
interpret tissue stained with haematoxylin and eosin (H&E), where haematoxylin
stains cell nuclei blue and eosin stains other cellular structures, such as
cytoplasm and cell membranes, varying shades of pink and red. The
interpretation of H&E slides can be a subjective, time-consuming process, with
some tasks having a high level of inter-observer variation [7, 8, 9]. In the
assessment of difficult cases, general pathologists may seek assistance from
subspecialty gynaecological pathology experts, and/or use ancillary tests,
such as immunohistochemical (IHC) stains. IHC stains indicate the presence of
specific antigens and are often used to aid pathologists in identifying the
primary tissue of origin or to make subtype diagnoses where there are specific
phenotypic profiles [5]. Referrals and ancillary testing can be essential to
the accuracy of the diagnostic process but come at the cost of making it
longer and more expensive. Worldwide, pathologists are in much greater demand
than supply, with significant disparities in the number of pathologists
between countries [10], and even better-supplied countries unable to meet
demand [11].

Traditionally, pathologists have analysed glass slides using a light
microscope. However, the implementation of a digital workflow, where
pathologists review scanned whole slide images (WSIs) using a computer, is
becoming more common. While digital pathology uptake has likely been driven by
efficiency benefits [12], it has created an opportunity for the development of
automated tools to assist pathologists. These tools often aim to improve the
accuracy, efficiency, objectivity, and consistency of diagnosis. Such tools
could help to alleviate the global workforce shortage of pathologists,
increasing diagnostic throughput and reducing the demand for referrals and
ancillary tests. This is an increasingly active area of research [13] and, for
some malignancies, these systems are starting to achieve clinical utility
[14].

In this study, we systematically reviewed all literature in which artificial
intelligence (AI) techniques (comprising both traditional machine learning
(ML) and deep learning (DL) methods) were applied to digital pathology images
for the diagnosis or prognosis of ovarian cancer. This included research which
focused on a single diagnostic factor such as histological subtype, and
studies that performed computer-aided diagnostic tasks such as tumour
segmentation. The review characterises the state of the field, describing
which diagnostic and prognostic tasks have been addressed, and assessing
factors relevant to the clinical utility of these methods, such as the risks
of bias. Despite ovarian cancer being a particularly difficult disease to
detect and diagnose, and the shortage of available pathologists, AI models
have not yet been implemented in clinical practice for this disease. This
review aims to provide insights and recommendations based on published
literature to improve the clinical utility of future research, including
reducing risks of bias, improving reproducibility, and increasing
generalisability.

## Methods

### Literature Search

Searches were conducted in three research databases, PubMed, Scopus and Web of
Science, and two trial registries, Cochrane Central Register of Controlled
Trials (CENTRAL) and the World Health Organisation International Clinical
Trial Registry Platform (WHO-ICTRP). The initial searches were performed on
25/04/2022 and were repeated on 01/12/2022. The search strategy was composed
of three distinct aspects - artificial intelligence, ovarian cancer, and
histopathology. For each aspect, multiple relevant terms were combined using
the _OR_ operator (e.g. ”artificial intelligence” OR ”machine learning”), and
then these were combined using the _AND_ operator to ensure that retrieved
research met all three aspects. The widest possible set of search fields was
used for each search engine except for Scopus, where restrictions were imposed
to avoid searching within the citation list of each article, which is not an
available field in the other search engines. The terms ’ML’ and ’AI’ were
restricted to specific fields due to the diversity of their possible meanings.
To ensure the most rigorous literature search possible, no restrictions were
placed on the publication date or article type during searching.

Many AI approaches build on statistical models, such as logistic regression,
which can blur the lines between disciplines. When conducting searches, a
previously reported methodology was adopted [15] whereby typical AI approaches
were searched by name (e.g. neural networks), and other methods were searched
by whether the authors described their work as _artificial intelligence_. Full
details of the search implementation for each database are provided in
Appendix A. The review protocol was registered with PROSPERO before the search
results were screened for inclusion (CRD42022334730).

### Literature Selection

One researcher (JB) manually removed duplicate papers with the assistance of
the referencing software _EndNote X9_. Two researchers (JB, KA) then
independently screened articles for inclusion in two stages, the first based
on title and abstract, the second based on full text. Disagreements were
discussed and arbitrated by a third researcher (NR). Trials in WHO-ICTRP do
not have associated abstracts, so for these studies, only titles were
available for initial screening.

The inclusion criteria required that research evaluated the use of at least
one AI approach to make diagnostic or prognostic inferences on human
histopathology images from suspected or confirmed cases of ovarian cancer.
Studies were only included where AI methods were applied directly to the
digital pathology images, or to features which were automatically extracted
from the images. Fundamental tasks such as segmentation and cell counting were
considered to be diagnostic tasks because these could be used by pathologists
for computer-assisted diagnosis. Only conventional light microscopy images
were considered, with other imaging modalities, such as fluorescence and
hyperspectral imaging, excluded. Publications which did not include primary
research were excluded (such as review papers). Non-English language articles
and research where a full version of the manuscript was not accessible were
excluded.

### Risk of Bias Analysis

The risk of bias of models in the accepted literature was assessed using the
Prediction model Risk Of Bias ASsessment Tool (PROBAST) [16]. This tool
includes 20 questions which are answered as either _yes_ , _probably yes_ ,
_probably no_ , _no_ , or _no information_. These questions are categorised
into four domains (participants, predictors, outcome, and analysis), which are
summarised as high-risk, low-risk, or unclear. An overall score is calculated
by aggregation of these domain-specific scores, with a single high-risk domain
being sufficient for an overall high-risk score. Each model was analysed by
three independent researchers (any of JB, KA, NR, KZ, NMO), with at least one
computer scientist and one clinician involved in the risk of bias assessment
for each model. The PROBAST applicability of research analysis was not
implemented as it is unsuitable for such a diverse array of possible research
questions.

### Data Synthesis

Data extraction was performed independently by two researchers (JB, KA) using
a form containing 81 fields within the categories _Overview_ , _Data_ ,
_Methods_ , _Results_ , and _Miscellaneous_. Several of these fields were
added or clarified during data extraction with the agreement of both
researchers and retroactively applied to all accepted literature. The final
data extraction form is available at www.github.com/scjjb/OvCaReview, with a
summary included in Appendix B.

Information was sought from full-text articles, as well as references and
supplementary materials where appropriate. Inferences were made only when both
researchers were confident that this gave the correct information, with
disagreements resolved through discussion. Fields which could not be
confidently completed were labelled as being _unclear_. Information was
extracted regarding each outcome reported in a paper for which the
corresponding model met the inclusion criteria. Where multiple models were
compared for the same outcome, data was only extracted for the newly proposed
model, with the best performing model during validation taken if this was
unclear. Models used to predict different outcomes in the same study were
assessed independently even if the methods were similar.

Data synthesis excluded any model which was not applied to ovarian cancer
digital pathology slides, such as repeats of the same methodology applied to
different malignancies. Models that met the inclusion criteria are referred to
as _models of interest_. All extracted data are summarised in two tables, one
each for study-level and model-level characteristics, with the model-level
table grouped by outcome type. The data synthesis did not include any meta-
analysis due to the diversity of included methods and outcomes.

## Results

Figure 1: PRISMA 2020 flowchart of the study identification and selection
process for the systematic review. Records were screened on titles and
abstracts alone, and reports were assessed based on the full-text content.

As shown in Figure 1, the literature searches returned a total of 1434
records, of which 496 were duplicates. 866 records were excluded during the
screening of titles and abstracts, and 36 were excluded based on full paper
screening, including 2 records for which full articles could not be obtained.
The remaining 36 studies were included in the review, of which 11 were
conference papers and 25 were journal papers. All accepted studies were
originally identified through searches of research databases, with no records
from trial registries meeting the inclusion criteria. While the searches
returned literature from as early as 1949, all of the research which met the
inclusion criteria was published since 2010, and over half of the included
literature was published since 2020. Study characteristics are shown in Table
2. The 36 accepted articles contained 62 models of interest, details of which
are shown in Table 3.

### Risk of Bias Analysis

Figure 2: PROBAST risk of bias results summarised for the 36 papers included
in this review.

The results of the PROBAST assessments are shown in Table 1. While some
studies contained multiple models of interest, none of these contained models
with different risk of bias scores for any section of the PROBAST assessment,
so we only present one risk of bias analysis per paper. All models showed
either a high overall risk of bias (30/36) or an unclear overall risk of bias
(6/36). Every high-risk model had a high-risk score in the analysis section
(30/36), with several also being at high risk for participants (5/36),
predictors (10/36), or outcomes (11/36). Only half of the studies achieved a
low risk of bias in any domain (18/36), with most low risks being found in the
outcomes (14/36) and predictors (8/36) sections. Nearly all of the papers had
an unclear risk of bias in at least one domain, most commonly the participants
(29/36) and predictors (18/36) domains. Qualitative summaries are presented in
Figure 2.

gray!20 Publication | Participants | Predictors | Outcome | Analysis |  Overall
---|---|---|---|---|---
Dong 2010(a) [17] |  red!25High |  red!25High |  red!25High |  red!25High |  red!25High
Dong 2010(b) [18] |  red!25High |  red!25High |  red!25High |  red!25High |  red!25High
Signolle 2010 [19] |  yellow!25Unclear |  yellow!25Unclear |  red!25High |  red!25High |  red!25High
Janowczyk 2011 [20] |  yellow!25Unclear |  yellow!25Unclear |  green!25Low |  red!25High |  red!25High
Janowczyk 2012 [21] |  yellow!25Unclear |  red!25High |  yellow!25Unclear |  red!25High |  red!25High
Kothari 2012 [22] |  yellow!25Unclear |  green!25Low |  green!25Low |  yellow!25Unclear |  yellow!25Unclear
Poruthoor 2013 [23] |  yellow!25Unclear |  red!25High |  red!25High |  red!25High |  red!25High
BenTaieb 2015 [24] |  yellow!25Unclear |  yellow!25Unclear |  green!25Low |  red!25High |  red!25High
BenTaieb 2016 [25] |  yellow!25Unclear |  red!25High |  yellow!25Unclear |  red!25High |  red!25High
BenTaieb 2017 [26] |  yellow!25Unclear |  yellow!25Unclear |  green!25Low |  red!25High |  red!25High
Lorsakul 2017 [27] |  yellow!25Unclear |  yellow!25Unclear |  red!25High |  red!25High |  red!25High
Du 2018 [28] |  yellow!25 Unclear |  yellow!25 Unclear |  yellow!25 Unclear |  yellow!25 Unclear |  yellow!25 Unclear
Heindl 2018 [29] |  yellow!25Unclear |  green!25Low |  green!25Low |  red!25High |  red!25High
Kalra 2020 [30] |  yellow!25Unclear |  green!25Low |  green!25Low |  red!25High |  red!25High
Levine 2020 [31] |  yellow!25Unclear |  green!25Low |  green!25Low |  yellow!25Unclear |  yellow!25Unclear
Yaar 2020 [32] |  yellow!25Unclear |  yellow!25Unclear |  green!25Low |  red!25High |  red!25High
Yu 2020 [33] |  yellow!25Unclear |  green!25Low |  green!25Low |  red!25High |  red!25High
Gentles 2021 [34] |  red!25High |  yellow!25Unclear |  red!25High |  red!25High |  red!25High
Ghoniem 2021 [35] |  yellow!25Unclear |  yellow!25Unclear |  yellow!25Unclear |  red!25High |  red!25High
Jiang 2021 [36] |  red!25High |  red!25High |  yellow!25Unclear |  red!25High |  red!25High
Laury 2021 [37] |  green!25Low |  red!25High |  red!25High |  red!25High |  red!25High
Paijens 2021 [38] |  green!25Low |  red!25High |  yellow!25Unclear |  red!25High |  red!25High
Shin 2021 [39] |  yellow!25Unclear |  yellow!25Unclear |  yellow!25Unclear |  red!25High |  red!25High
Zeng 2021 [40] |  yellow!25Unclear |  yellow!25Unclear |  green!25Low |  red!25High |  red!25High
Boehm 2022 [41] |  yellow!25Unclear |  red!25High |  yellow!25Unclear |  red!25High |  red!25High
Boschman 2022 [42] |  yellow!25Unclear |  green!25Low |  green!25Low |  red!25High |  red!25High
Elie 2022 [43] |  yellow!25Unclear |  green!25Low |  red!25High |  red!25High |  red!25High
Farahani 2022 [44] |  yellow!25Unclear |  yellow!25Unclear |  green!25Low |  yellow!25Unclear |  yellow!25Unclear
Hu 2022 [45] |  yellow!25Unclear |  yellow!25Unclear |  yellow!25Unclear |  yellow!25Unclear |  yellow!25Unclear
Jiang 2022 [46] |  yellow!25Unclear |  yellow!25Unclear |  red!25High |  red!25High |  red!25High
Kasture 2022 [47] |  red!25High |  red!25High |  red!25High |  red!25High |  red!25High
Kowalski 2022 [48] |  yellow!25Unclear |  yellow!25Unclear |  yellow!25Unclear |  red!25High |  red!25High
Liu 2022 [49] |  yellow!25Unclear |  yellow!25Unclear |  yellow!25Unclear |  yellow!25Unclear |  yellow!25Unclear
Nero 2022 [50] |  yellow!25Unclear |  green!25Low |  red!25High |  red!25High |  red!25High
Salguero 2022 [51] |  yellow!25Unclear |  yellow!25Unclear |  green!25Low |  red!25High |  red!25High
Wang 2022 [52] |  yellow!25Unclear |  yellow!25Unclear |  green!25Low |  red!25High |  red!25High
Table 1: PROBAST risk of bias assessment results for the 36 papers included in
this review. This is presented as one row for each paper because every paper
that contained multiple models of interest was found to have the same risk of
bias for every model.

### Data Synthesis Results

Table 2: Characteristics of the 36 studies included in this systematic review.
Details are shown for individual models in Table 3. Four data sources are used
in multiple studies - The Cancer Genome Atlas (TCGA-OV) [53], the British
Columbia Ovarian Cancer Research Program (OVCARE), The Transcanadian Study
[54], and the Mayo Clinic. Code is labelled as inaccessible where it could not
be found despite a link being provided in the publication. H&E - haematoxylin
and eosin. IHC - immunohistochemical. WSI - whole slide image. TMA - tissue
microarray.

Table 3: Characteristics of the 62 models of interest from the 36 papers
included in this systematic review, grouped by outcome type. *Other data types
are Genomics (G), Proteomics (P), Radiomics (R), and Transcriptomics (T). SVM
- support vector machine. CNN - convolutional neural network. AUC - area under
the receiver operating characteristic (ROC) curve. HGSC - high-grade serous
carcinoma. LGSC - low-grade serous carcinoma. CCC - clear cell carcinoma. MC -
mucinous carcinoma. EC - endometrioid carcinoma.

#### Data in Included Literature

The number of participants in internal datasets varied by orders of magnitude,
with each study including 1 to 664 ovarian cancer patients, and one study
including over 10,000 total patients across a range of 32 malignancies [30].
Only the five most common subtypes of ovarian carcinoma were used, with no
study reporting the inclusion of less common carcinomas or non-epithelial
ovarian cancers. Only one study explicitly included any prospective data
collection, and this was only for a small subset which was not used for
external validation [41].

As shown in Figure 3, the number of pathology slides used was often much
greater than the number of patients included, with three studies using over
1000 slides from ovarian cancer patients [22, 33, 49]. Most of the studies
used WSIs for model development (27/36), with others using tissue microarrays
(TMAs) (4/36) or pre-cropped digital pathology images (2/36). Most studies
used H&E-stained tissue (27/36) and the others used a variety of IHC stains
(9/36), with no two papers reporting the use of the same IHC stains. Some
studies included multi-modal approaches, using genomics [23, 32, 35, 40, 41],
proteomics [23, 40], transcriptomics [40], and radiomics [41] data alongside
histopathological data.

The most commonly used data source was The Cancer Genome Atlas (TCGA) (14/36),
a project from which over 30,000 digital pathology images from 33 malignancies
are publicly available. The ovarian cancer subset, TCGA-OV [53], contains 1481
WSIs from 590 cases of ovarian serous carcinoma (mostly, but not exclusively,
high-grade), with corresponding genomic, transcriptomic, and clinical data.
This includes slides from eight data centres in the United States, with most
slides containing frozen tissue sections (1374/1481) rather than formalin-
fixed, paraffin-embedded (FFPE) sections. Other recurring data sources were
the University of British Columbia Ovarian Cancer Research Program (OVCARE)
repository [31, 42, 44], the Transcanadian study [24, 25], and the Mayo Clinic
records [36, 46], each of which was used in multiple publications by a single
research group. All other researchers either used a unique data source (11/36)
or did not report the provenance of their data (8/36). TCGA-OV, OVCARE, and
the Transcanadian study are all multi-centre datasets. Aside from these, few
studies reported the use of multi-centre data [38, 39, 40, 41, 44]. Only two
studies reported the use of multiple slide scanners, with every slide scanned
on one of two available scanners [42, 44]. The countries from which data were
sourced included Canada, China, Finland, France, Italy, the Netherlands, South
Korea, Taiwan, the United Kingdom, and the United States of America.

Figure 3: Histograms showing the number of ovarian cancer patients and slides
used in model development. Many of these values are uncertain due to
incomplete reporting, as reflected in Table 3.

#### Methods in Included Literature

There was a total of 62 models of interest in the 36 included papers, with
each paper containing 1 to 8 such models. These models consisted of 35
classifiers, 14 survival prediction models, 7 segmentation models, and 6
regression models. A variety of classification outcomes were assessed -
histological subtype (7/35), malignancy (5/35), primary cancer type (4/35),
genetic mutation status (3/35), stain intensity (3/35), tumour grade (2/35),
tissue type (2/35), cell type (2/35), microsatellite instability (2/35),
transcriptomic subtype (2/35), stage (1/35), epithelial-mesenchymal transition
status (1/35), and treatment response (1/35). Most survival models measured
overall survival (9/14), while others measured progression-free survival
(2/14), platinum-free interval (2/14) and symptom-free interval (1/14).
Segmentation models were split between tumour segmentation (4/7) and stain
segmentation (3/7). The regression models also quantified staining but were
formulated as regression tasks rather than segmentation.

A variety of models were used, with the most common types being convolutional
neural network (CNN) (21/62), support vector machine (SVM) (10/62), and random
forest (9/62). CNN architectures included GoogLeNet [28], VGG16 [33], VGG19
[31, 44], InceptionV3 [39], ResNet18 [42, 45], ResNet50 [50], and MaskRCNN
[46]. Novel CNNs typically used multiple standardised blocks involving
convolutional, normalization, activation, and/or pooling layers [32, 47, 48],
with one study also including attention modules in these blocks [49]. One
study generated their novel architecture by using a topology optimization
approach on a standard VGG16 [35].

Most researchers split their original images into patches to be separately
processed, with patch sizes ranging from 60x60 to 2048x2048 pixels, the most
common being 256x256 pixels (6/36) and 512x512 pixels (5/36). A range of
feature extraction techniques were employed, with a nearly even split between
hand-crafted/pre-defined features (26/62) and features that were automatically
learned by the model (30/62). Hand-crafted features included a plethora of
textural, chromatic, and cellular and nuclear morphological features. Hand-
crafted features were commonly used as inputs to classical ML methods, such as
SVM and random forest models. Learned features were typically extracted using
a CNN, which was often also used for classification.

Despite the common use of patches, most models made predictions at the WSI
level (25/62) or patient level (11/62), requiring aggregation of patch-level
information. Two distinct aggregation approaches were used, one aggregating
before modelling and one aggregating after modelling. The former approach
requires the generation of slide-level features before modelling, the latter
requires the aggregation of patch-level model outputs to make slide-level
predictions. Slide-level features were generated using averaging [23, 40],
attention-based weighted averaging [45, 49, 50], concatenating [25, 30], as
well as more complex embedding approaches using Fisher vector encoding [24]
and k-means clustering [26]. Patch-level model outputs were aggregated to
generate slide-level predictions by taking the maximum [32] or average [35],
using voting strategies [42, 52], or using a random forest classifier [44].
These approaches are all examples of _multiple instance learning_ (MIL),
though few models of interest were reported using this terminology [32, 45,
50].

Despite attention-based approaches having been applied to other malignancies
for several years [55, 56], they were only seen in the most recent ovarian
cancer studies [44, 45, 49, 50, 52], and none of the methods included self-
attention, an increasingly popular method for other malignancies [57]. Most
models were deterministic, though hidden Markov trees [19], probabilistic
boosting trees [20], and Gaussian mixture models [43] were also used. Tissue
was typically analysed at a single resolution, with only four papers including
multi-magnification techniques in their models of interest. Two of these
combined features from different resolutions for modelling [24, 26], and the
other two used low-resolution images to determine areas of interest in high-
resolution images [25, 52]. Out of the papers for which it could be
determined, the most common modelling magnifications were 20x (26/31) and 40x
(7/31). Few models integrated histopathology data with other modalities
(8/62). Multi-modal approaches included the concatenation of separately
extracted uni-modal features before modelling [23, 35, 40], the amalgamation
of uni-modal predictions from separate models [41], and a teacher-student
approach where multiple modalities were used in model training but only
histopathology data was used for prediction [32].

#### Analysis in Included Literature

Analyses were limited, with less than half of the outcomes being evaluated
with cross-validation (24/62) and/or external validation on independent
ovarian cancer datasets (7/62) despite small internal cohort sizes. Cross-
validation methods included k-fold (11/24) with 4 to 10 folds, Monte Carlo
(8/24) with 3 to 15 repeats, and leave-one-patient-out cross-validations
(5/24). Some other papers included cross-validation on the training set to
select hyperparameters but used only a small unseen test set from the same
data source for evaluation. Externally validated models were all trained with
WSIs, with validations either performed on TMAs (4/7) or WSIs from independent
data sources (3/7), with two of these explicitly using different scanners to
digitize internal and external data [42, 44]. Some papers included external
validation with different malignancies, but none of these included ovarian
cancer data in any capacity.

Most classification models were evaluated using accuracy, balanced accuracy,
and/or area under the receiver operating characteristic curve (AUC), with one
exception where only a p-value was reported measuring the association between
histological features and transcriptomic subtypes based on a Kruskal-Wallis
test [33]. Some models were also evaluated using the F1-score, which we chose
not to tabulate (in Figure 3) as the other metrics were reported more
consistently. Survival model performance was reported using AUC, p-value,
accuracy and hazard ratios. Segmentation models were almost all evaluated
differently from each other, with different studies reporting AUC, accuracy,
Dice coefficient, sensitivity, specificity, and qualitative evaluations.
Regression models were all evaluated using the coefficient of determination
($R^{2}$-statistic).

The variability of model performance was not frequently reported (20/78), and
when it was reported it was often incomplete. This included cases where it was
unclear what the intervals represented (95% confidence interval, one standard
deviation, variation, etc.), or not clear what the exact bounds of the
interval were due to results being plotted but not explicitly stated. Within
the entire review, there were only two examples in which variability was
reported during external validation [39, 42], one of which did not clearly
state either the bounds or the type of the interval. No studies performed any
Bayesian form of uncertainty quantification. Reported results are shown in
Table 3, though direct comparisons between the performance of different models
should be treated with caution due to the diversity of data and validation
methods used to evaluate different models, the lack of variability measures,
the consistently high risks of bias, and the heterogeneity in reported
metrics.

## Discussion

The vast majority of published research on AI for diagnostic/prognostic
purposes in ovarian cancer histopathology was found to be at a high risk of
bias due to issues within the analyses performed. Researchers often used a
limited quantity of data or did not include sufficient validation to account
for overfitting and model optimism (cross-validation, bootstrapping, external
validation) within their study methodology. While data quantity may have been
limited by technical and financial constraints, the lack of thorough
validation is a key issue which can be corrected regardless of other
limitations through improved study design.

The more robust analyses included one study in which several relevant metrics
were evaluated using 10 repeats of Monte Carlo cross-validation on a set of
406 WSIs, with standard deviations reported for each metric [31]. Another
positive example included the use of both an internal five-fold cross-
validation, and an external validation for the same outcome, giving a more
rigorous analysis [52]. While external validations were uncommon, those which
were conducted offered a real insight into model generalisability, with a
clear reduction in performance on all external validation sets except one
[44]. The only study which demonstrated high generalisability included the
largest training set out of all externally validated approaches, included more
extensive data labelling than many similar studies, and implemented a
combination of three colour normalisation approaches, indicating that these
factors may benefit generalisability.

Studies frequently had an unclear risk of bias within the participants (29/36)
and predictors (18/36) domains of PROBAST, with published work rarely
reporting information about patient recruitment and inclusion, especially when
using open-access datasets. Only two papers were found to be at low risk of
bias for participants, with these including clear and reasonable patient
recruitment strategies and selection criteria, which can be seen as positive
examples for other researchers [37, 38]. Information about the predictors
(histopathology images and features derived thereof) was generally better
reported, but still often missed key details which meant that it was unclear
whether all tissue samples were processed similarly to avoid risks of bias
from visual heterogeneity.

It was found that when patient characteristics were reported, they often
showed a high risk of bias. Many studies included very small numbers of
patients with specific differences from the majority - for example, a minority
where specimens were processed with a different staining protocol, leading to
variable image appearance. This can be a source of bias because the minority
subgroup may be correlated with the outcome of interest by chance, so a model
can make predictions based on a surrogate marker which may only be useful in
one specific dataset, and is not generalisable to the wider population. Such a
surrogate marker may have little to do with the outcome of interest, being the
result of a spurious correlation in the data learned by the model. Larger
population subgroups can also cause bias, though this is less likely to be
caused by random chance and more likely to be influenced by structural
confounding factors. One paper was also found to have major discrepancies
between the reported data, the study design, and the data that was available
through a link in the paper, indicating a significant risk of bias [47]. In
this case, it was reported that TCGA-OV data was used for multi-class
subtyping, despite this dataset only including high-grade serous and low-grade
serous carcinomas.

### Limitations of the Review

The main limitation of this review is the restriction to the English language
- AI research is a global field, and relevant literature has likely been
published in other languages, making this review incomplete. While most of the
review process was completed by multiple independent researchers, the
duplicate detection was performed by only a single researcher, raising the
possibility of errors in this step of the review process, resulting in
incorrect exclusions. Due to the significant time gap between the initial and
final literature searches (approximately 7 months), there may have been
inconsistencies in interpretations, both for data extraction and risk of bias
assessments. Finally, this review focused only on light microscopy images of
human histopathology samples relating to ovarian cancer, so may have
overlooked useful literature outside of this domain.

### Development of the Field

The field of AI in ovarian cancer histopathology diagnosis is rapidly growing,
with more research published since the start of 2020 than in all preceding
years combined. The earliest research, published between 2010-2013, used hand-
crafted features to train classical ML methods such as SVMs. These models were
used for segmentation [17, 18, 19, 20, 21], malignancy classification [22],
grading [23], and survival prediction [23]. Most of these early studies
focused on IHC-stained tissue (5/7), which would be much less commonly used in
subsequent research (4/29).

The field was relatively dormant in the following years, with only 6 papers
published between 2014-2019, half of which had the same primary author [24,
25, 26]. These models still used traditional ML classifiers, though some used
learned features rather than the traditional hand-crafted features. The models
developed were used for histological subtyping [24, 25, 26] and
cellular/tissue classification [27, 28, 29].

Since 2020 there has been a much greater volume of research published, most of
which has involved the use of deep neural networks for automatic feature
extraction and classification. Recent research has investigated a broader
array of outcomes, including the classification of primary cancer type [30],
mutation status [40, 50], transcriptomic subtypes [33, 40], microsatellite
instability [40], epithelial-mesenchymal transition status [45], and treatment
response prediction [52]. Three additional survival outcomes have also been
predicted in more recent literature - symptom-free interval [32], platinum-
free interval [33, 37], and progression-free survival [41, 50].

Despite progress within a few specific outcomes, there was no obvious overall
trend in the sizes of datasets used over time, either in terms of the number
of slides or the number of participants. Similarly, there was no evidence that
recent research included more rigorous internal validations, though external
validations have been increasing in frequency - no research before 2021
included any external validation with ovarian cancer data, but two papers
published in 2021 [39, 40] and three published in 2022 [42, 44, 52] did. These
external validations were typically limited to small quantities of data from a
single external data centre or of a different data type (TMA rather than WSI).
However, the inclusion of any external validation demonstrates progress from
previous research. Such validations are essential to the clinical utility of
these models as real-world implementation will require robustness to different
sources of visual heterogeneity, with variation occurring across different
data centres and within data centres over time. As this field continues to
mature, we hope to see more studies conduct thorough validations with larger,
high-quality independent datasets, including clearly reported protocols for
patient recruitment and selection, pathology slide creation, and digitization.
This will help to reduce the biases, limited reproducibility, and limited
generalisability identified in most of the existing research in this domain.

### Current Limitations and Future Recommendations

A large proportion of published work did not provide sufficient clinical and
pathological information to assess the risk of bias. Common types of missing
information included where the patients were recruited, how many patients were
included, how many samples/images were used, whether any patients/images were
excluded, and the methods by which tissue was processed and digitized. The
latter includes details about the fixing, staining, and scanning of tissue,
processes which are likely causes of visual heterogeneity in pathology slides.
This heterogeneity can lead to confounding or bias in models when not properly
accounted for, especially when using small datasets where random correlations
between unrelated factors are more likely to occur. When using sufficiently
large datasets and rigorous methodologies to account for confounding, visual
heterogeneity can be beneficial as models can be trained to account for these
variations. To understand the effects of heterogeneity it is important that AI
researchers thoroughly report data provenance. Researchers may find it useful
to refer to reporting checklists, such as _transparent reporting of a
multivariable prediction model for individual prognosis or diagnosis_
(TRIPOD), to ensure that they have understood and reported all relevant
details of their studies.

Reporting was particularly sparse in studies which used openly accessible
data, possibly indicating that AI-focused researchers were not taking
sufficient time to understand these datasets and ensure their research was
clinically relevant. For example, many of the researchers who used TCGA data
included frozen tissue sections without commenting on whether this was
appropriate, despite the fact that pathologists do not consider them to be of
optimal diagnostic quality. One paper handled TCGA data more appropriately,
with a clear explanation of the positives and negatives of the dataset, and
entirely separate models for FFPE and frozen slides [30]. AI researchers
should seek to understand the clinical context of their data before
undertaking research to reduce bias and increase clinical utility. Ideally,
this should involve regular interactions with expert clinicians, including
histopathologists and oncologists.

Many researchers reported results from only a single train-test split of their
data, which raises questions about the reliability of results, especially with
small datasets. We recommend that researchers should always conduct more
thorough analyses, using cross-validation, bootstrapping, and/or external
validations to ensure that results are robust and truly reflect the ability of
their model(s) to generalise to unseen data, and are not simply caused by
chance. It is also beneficial to report the variability of results (typically
in a 95% confidence interval), especially when comparing multiple models,
where confidence intervals can help to distinguish whether one model is
genuinely better than another or whether the difference is due to chance.
Statistical tests can also be beneficial for these evaluations. Another option
for capturing variability is Bayesian uncertainty quantification, which can be
used to separate aleatoric (inherent) and epistemic (modelling) uncertainty.

The incomplete reporting observed in many studies makes them much less
reproducible. As well as the previously mentioned factors around patient
recruitment and data processing, there was often missing information about AI
methodology and analysis approaches. The negative effect that incomplete
reporting has on reproducibility can be significantly mitigated by publishing
code and data. Only 14 of the 36 included papers made any attempt to share
code, with some of these appearing to be incomplete or inaccessible. The
better code repositories included detailed documentation to aid
reproducibility, including environment set-up information [33, 42], overviews
of included functions [41], and code examples used to generate reported
results [29]. It is relatively easy to publish code and generate documentation
to enhance usability, and there are few drawbacks to doing so when publishing
research. Making data available is more difficult due to data security
requirements and the potential storage costs, but it can provide benefits
beyond the primary research of the original authors. Digital pathology
research in ovarian cancer is currently limited by the lack of openly
accessible data, leading to over-dependence on TCGA, and causing many
researchers to painstakingly collate similar but distinct datasets. These
datasets often contain little of the heterogeneity seen in multi-centre,
multi-scanner data, making it difficult for researchers to train robust models
or assess generalisability. Making more data openly accessible, with detailed
protocols describing data creation, would allow future researchers to conduct
more thorough analyses and subsequently improve model generalisability and
clinical implementability.

Current literature in this field can be largely characterised as model
prototyping with homogeneous retrospective data. Studies rarely consider the
reality of human-machine interaction, perhaps believing that these models are
a drop-in replacement for pathologists. However, these models perform narrow
tasks within the pathology pipeline and have no understanding of context
beyond their limited training datasets. We believe these models would be more
beneficial (and more realistic to implement) as assistive tools for
pathologists, providing secondary opinions or novel ancillary information.
While current research is typically focused on assessing model accuracy
without any pathologist input, different study designs could be employed to
better assess the real-world utility of these models as assistive tools. For
example, usability studies could investigate which models are most accessible
and most informative to pathologists in practice, and prospective studies
could quantify any benefits to diagnostic efficiency and patient outcomes, and
investigate the robustness of models in practice. Understanding the effects of
AI on the efficiency of diagnosis is particularly important given the limited
supply of pathologists worldwide. As such, this type of research will
significantly benefit clinical translation.

### Summary of recommendations

  * •

Understand data and ensure planned research is clinically relevant before
modelling, ideally involving clinicians throughout the project.

  * •

Consider different study designs, including usability studies and/or
prospective studies

  * •

Clearly report the context of any histopathology data, including how patients
were recruited/selected, and how tissue specimens were processed to generate
digital pathology images.

  * •

Conduct thorough analyses using cross-validation, external validation, and/or
bootstrapping.

  * •

Make all code openly accessible (and data if possible).

## Acknowledgments

There was no direct funding for this research. JB is supported by the UKRI
Engineering and Physical Sciences Research Council (EPSRC) [EP/S024336/1]. KA,
PA are supported by the Tony Bramall Charitable Trust. AS is supported by
Innovate UK via the National Consortium of Intelligent Medical Imaging (NCIMI)
[104688], Cancer Research UK [C19942/A28832] and Leeds Hospitals Charity
[9R01/1403]. The funders had no role in influencing the content of this
research. For the purpose of open access, the author has applied a Creative
Commons Attribution (CC BY) licence to any Author Accepted Manuscript version
arising from this submission.

## Author Contributions

JB created the study protocol with feedback and contributions from all other
authors. JB, KA, KZ, NMO, and NR performed the risk of bias assessments. JB
and KA performed data extraction. JB analysed extracted data and wrote the
manuscript, with feedback and contributions from all other authors.

## Competing Interests

GH receives research funding from IQVIA. NMO receives research funding from 4D
Path. All other authors declare no conflicts of interest.

## References

  * [1] Hyuna Sung, Jacques Ferlay, Rebecca L. Siegel, Mathieu Laversanne, Isabelle Soerjomataram, Ahmedin Jemal, and Freddie Bray.  Global cancer statistics 2020: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries.  CA: A Cancer Journal for Clinicians, 71, 2021.
  * [2] Usha Menon, Aleksandra Gentry-Maharaj, Matthew Burnell, Naveena Singh, Andy Ryan, Chloe Karpinskyj, Giulia Carlino, Julie Taylor, Susan K. Massingham, Maria Raikou, Jatinderpal K. Kalsi, Robert Woolas, Ranjit Manchanda, Rupali Arora, Laura Casey, Anne Dawnay, Stephen Dobbs, Simon Leeson, Tim Mould, Mourad W. Seif, Aarti Sharma, Karin Williamson, Yiling Liu, Lesley Fallowfield, Alistair J. McGuire, Stuart Campbell, Steven J. Skates, Ian J. Jacobs, and Mahesh Parmar.  Ovarian cancer population screening and mortality after long-term follow-up in the uk collaborative trial of ovarian cancer screening (ukctocs): a randomised controlled trial.  The Lancet, 397, 2021.
  * [3] Mark H. Ebell, Marybeth B. Culp, and Taylor J. Radke.  A systematic review of symptoms for the diagnosis of ovarian cancer.  American Journal of Preventive Medicine, 50, 2016.
  * [4] Jonathan S. Berek, Malte Renz, Sean Kehoe, Lalit Kumar, and Michael Friedlander.  Cancer of the ovary, fallopian tube, and peritoneum: 2021 update.  International Journal of Gynecology and Obstetrics, 155, 2021.
  * [5] Martin Köbel, Steve E. Kalloger, Niki Boyd, Steven McKinney, Erika Mehl, Chana Palmer, Samuel Leung, Nathan J. Bowen, Diana N. Ionescu, Ashish Rajput, Leah M. Prentice, Dianne Miller, Jennifer Santos, Kenneth Swenerton, C. Blake Gilks, and David Huntsman.  Ovarian carcinoma subtypes are different diseases: Implications for biomarker studies.  PLoS Medicine, 5, 2008.
  * [6] Jaime Prat.  Staging classification for cancer of the ovary, fallopian tube, and peritoneum.  International Journal of Gynecology and Obstetrics, 124:1–5, 2014\.
  * [7] Rayna K. Matsuno, Mark E. Sherman, Kala Visvanathan, Marc T. Goodman, Brenda Y. Hernandez, Charles F. Lynch, Olga B. Ioffe, David Horio, Charles Platz, Sean F. Altekruse, Ruth M. Pfeiffer, and William F. Anderson.  Agreement for tumor grade of ovarian carcinoma: Analysis of archival tissues from the surveillance, epidemiology, and end results residual tissue repository.  Cancer Causes and Control, 24, 2013.
  * [8] Martin Köbel, Julia Bak, Björn I. Bertelsen, Olli Carpen, Anni Grove, Estrid S. Hansen, Anne Marie Levin Jakobsen, Marianne Lidang, Anna Måsbäck, Anna Tolf, C. Blake Gilks, and Joseph W. Carlson.  Ovarian carcinoma histotype determination is highly reproducible, and is improved through the use of immunohistochemistry.  Histopathology, 64, 2014.
  * [9] Mollie E. Barnard, Alexander Pyden, Megan S. Rice, Miguel Linares, Shelley S. Tworoger, Brooke E. Howitt, Emily E. Meserve, and Jonathan L. Hecht.  Inter-pathologist and pathology report agreement for ovarian tumor characteristics in the nurses’ health studies.  Gynecologic Oncology, 150, 2018.
  * [10] Michael L. Wilson, Kenneth A. Fleming, Modupe A. Kuti, Lai Meng Looi, Nestor Lago, and Kun Ru.  Access to pathology and laboratory medicine services: a crucial gap.  The Lancet, 391, 2018.
  * [11] Royal College of Pathologists.  Meeting pathology demand: Histopathology workforce census.  2018\.
  * [12] Alexi Baidoshvili, Anca Bucur, Jasper van Leeuwen, Jeroen van der Laak, Philip Kluin, and Paul J. van Diest.  Evaluating the benefits of digital pathology implementation: time savings in laboratory logistics.  Histopathology, 73, 2018.
  * [13] Albrecht Stenzinger, Maximilian Alber, Michael Allgäuer, Philipp Jurmeister, Michael Bockmayr, Jan Budczies, Jochen Lennerz, Johannes Eschrich, Daniel Kazdal, Peter Schirmacher, Alex H. Wagner, Frank Tacke, David Capper, Klaus Robert Müller, and Frederick Klauschen.  Artificial intelligence and pathology: From principles to practice and future applications in histomorphology and molecular profiling.  Seminars in Cancer Biology, 84, 2022.
  * [14] Patricia Raciti, Jillian Sue, Juan A Retamero, Rodrigo Ceballos, Ran Godrich, Jeremy D Kunz, Adam Casson, Dilip Thiagarajan, Zahra Ebrahimzadeh, Julian Viret, et al.  Clinical validation of artificial intelligence–augmented pathology diagnosis demonstrates significant gains in diagnostic accuracy in prostate cancer detection.  Archives of Pathology & Laboratory Medicine, 2022.
  * [15] Paula Dhiman, Jie Ma, Constanza L. Andaur Navarro, Benjamin Speich, Garrett Bullock, Johanna A. A. Damen, Lotty Hooft, Shona Kirtley, Richard D. Riley, Ben Van Calster, Karel G. M. Moons, and Gary S. Collins.  Methodological conduct of prognostic prediction models developed using machine learning in oncology: a systematic review.  BMC Medical Research Methodology, 22:101, 12 2022.
  * [16] Robert F. Wolff, Karel G.M. Moons, Richard D. Riley, Penny F. Whiting, Marie Westwood, Gary S. Collins, Johannes B. Reitsma, Jos Kleijnen, and Sue Mallett.  Probast: A tool to assess the risk of bias and applicability of prediction model studies.  Annals of Internal Medicine, 170, 2019.
  * [17] Jiwen Dong, Jing Li, Jian Lu, and Aifang Fu.  Automatic segmentation for ovarian cancer immunohistochemical image based on chroma criterion.  In 2010 2nd International Conference on Advanced Computer Control, volume 2, pages 147–150. IEEE, 2010.
  * [18] Jiwen Dong, Jing Li, Aifang Fu, and Huiming Lv.  Automatic segmentation for ovarian cancer immunohistochemical image based on yuv color space.  In 2010 International Conference on Biomedical Engineering and Computer Science, pages 1–4. IEEE, 2010.
  * [19] Nicolas Signolle, Marinette Revenu, Benoît Plancoulaine, and Paulette Herlin.  Wavelet-based multiscale texture segmentation: Application to stromal compartment characterization on virtual slides.  Signal Processing, 90, 2010.
  * [20] Andrew Janowczyk, Sharat Chandran, Michael Feldman, and Anant Madabhushi.  Local morphologic scale: application to segmenting tumor infiltrating lymphocytes in ovarian cancer tmas.  In Medical Imaging 2011: Image Processing, volume 7962, pages 827–840. SPIE, 2011.
  * [21] Andrew Janowczyk, Sharat Chandran, Rajendra Singh, Dimitra Sasaroli, George Coukos, Michael D. Feldman, and Anant Madabhushi.  High-throughput biomarker segmentation on ovarian cancer tissue microarrays via hierarchical normalized cuts.  IEEE Transactions on Biomedical Engineering, 59, 2012.
  * [22] Sonal Kothari, John H Phan, Adeboye O Osunkoya, and May D Wang.  Biological interpretation of morphological patterns in histopathological whole-slide images.  In Proceedings of the ACM conference on bioinformatics, computational biology and biomedicine, pages 218–225, 2012.
  * [23] Anjaly Poruthoor, John H Phan, Sonal Kothari, and May D Wang.  Exploration of genomic, proteomic, and histopathological image data integration methods for clinical prediction.  In 2013 IEEE China Summit and International Conference on Signal and Information Processing, pages 259–263. IEEE, 2013.
  * [24] Aïcha BenTaieb, Hector Li-Chang, David Huntsman, and Ghassan Hamarneh.  Automatic diagnosis of ovarian carcinomas via sparse multiresolution tissue representation.  In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part I 18, pages 629–636. Springer, 2015.
  * [25] Aïcha BenTaieb, Masoud S Nosrati, Hector Li-Chang, David Huntsman, and Ghassan Hamarneh.  Clinically-inspired automatic classification of ovarian carcinoma subtypes.  Journal of pathology informatics, 7(1):28, 2016.
  * [26] Aïcha BenTaieb, Hector Li-Chang, David Huntsman, and Ghassan Hamarneh.  A structured latent model for ovarian carcinoma subtyping from histopathology slides.  Medical image analysis, 39:194–205, 2017.
  * [27] Auranuch Lorsakul, Emilia Andersson, Suzana Vega Harring, Hadassah Sade, Oliver Grimm, and Joerg Bredno.  Automated wholeslide analysis of multiplex-brightfield ihc images for cancer cells and carcinoma-associated fibroblasts.  In Medical Imaging 2017: Digital Pathology, volume 10140, pages 41–46. SPIE, 2017.
  * [28] Yue Du, Roy Zhang, Abolfazl Zargari, Theresa C. Thai, Camille C. Gunderson, Katherine M. Moxley, Hong Liu, Bin Zheng, and Yuchen Qiu.  Classification of tumor epithelium and stroma by exploiting image features learned by deep convolutional neural networks.  Annals of Biomedical Engineering, 46, 2018.
  * [29] Andreas Heindl, Adnan Mujahid Khan, Daniel Nava Rodrigues, Katherine Eason, Anguraj Sadanandam, Cecilia Orbegoso, Marco Punta, Andrea Sottoriva, Stefano Lise, Susana Banerjee, and Yinyin Yuan.  Microenvironmental niche divergence shapes brca1-dysregulated ovarian cancer morphological plasticity.  Nature Communications, 9, 2018.
  * [30] Shivam Kalra, H. R. Tizhoosh, Sultaan Shah, Charles Choi, Savvas Damaskinos, Amir Safarpoor, Sobhan Shafiei, Morteza Babaie, Phedias Diamandis, Clinton J.V. Campbell, and Liron Pantanowitz.  Pan-cancer diagnostic consensus through searching archival histopathology images using artificial intelligence.  npj Digital Medicine, 3, 2020.
  * [31] Adrian B. Levine, Jason Peng, David Farnell, Mitchell Nursey, Yiping Wang, Julia R. Naso, Hezhen Ren, Hossein Farahani, Colin Chen, Derek Chiu, Aline Talhouk, Brandon Sheffield, Maziar Riazy, Philip P. Ip, Carlos Parra-Herran, Anne Mills, Naveena Singh, Basile Tessier-Cloutier, Taylor Salisbury, Jonathan Lee, Tim Salcudean, Steven J.M. Jones, David G. Huntsman, C. Blake Gilks, Stephen Yip, and Ali Bashashati.  Synthesis of diagnostic quality cancer pathology images by generative adversarial networks.  Journal of Pathology, 252, 2020.
  * [32] Asfand Yaar, Amina Asif, Shan E Ahmed Raza, Nasir Rajpoot, and Fayyaz Minhas.  Cross-domain knowledge transfer for prediction of chemosensitivity in ovarian cancer patients.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 928–929, 2020.
  * [33] Kun Hsing Yu, Vincent Hu, Feiran Wang, Ursula A. Matulonis, George L. Mutter, Jeffrey A. Golden, and Isaac S. Kohane.  Deciphering serous ovarian carcinoma histopathology and platinum response by convolutional neural networks.  BMC Medicine, 18, 2020.
  * [34] Lucy Gentles, Rachel Howarth, Won Ji Lee, Sweta Sharma-Saha, Angela Ralte, Nicola Curtin, Yvette Drew, and Rachel Louise O’Donnell.  Integration of computer-aided automated analysis algorithms in the development and validation of immunohistochemistry biomarkers in ovarian cancer.  Journal of Clinical Pathology, 74, 2021.
  * [35] Rania M. Ghoniem, Abeer D. Algarni, Basel Refky, and Ahmed A. Ewees.  Multi-modal evolutionary deep learning model for ovarian cancer diagnosis.  Symmetry, 13, 2021.
  * [36] Jun Jiang, Burak Tekin, Ruifeng Guo, Hongfang Liu, Yajue Huang, and Chen Wang.  Digital pathology-based study of cell- and tissue-level morphologic features in serous borderline ovarian tumor and high-grade serous ovarian cancer.  Journal of Pathology Informatics, 12, 2021.
  * [37] Anna Ray Laury, Sami Blom, Tuomas Ropponen, Anni Virtanen, and Olli Mikael Carpén.  Artificial intelligence-based image analysis can predict outcome in high-grade serous carcinoma via histology alone.  Scientific Reports, 11, 2021.
  * [38] S. T. Paijens, A. Vledder, D. Loiero, E. W. Duiker, J. Bart, A. M. Hendriks, M. Jalving, H. H. Workel, H. Hollema, N. Werner, A. Plat, G. B.A. Wisman, R. Yigit, H. Arts, A. J. Kruse, N. M. de Lange, V. H. Koelzer, M. de Bruyn, and H. W. Nijman.  Prognostic image-based quantification of cd8cd103 t cell subsets in high-grade serous ovarian cancer patients.  OncoImmunology, 10, 2021.
  * [39] Seo Jeong Shin, Seng Chan You, Hokyun Jeon, Ji Won Jung, Min Ho An, Rae Woong Park, and Jin Roh.  Style transfer strategy for developing a generalizable deep learning application in digital pathology.  Computer Methods and Programs in Biomedicine, 198:105815, 2021.
  * [40] Hao Zeng, Linyan Chen, Mingxuan Zhang, Yuling Luo, and Xuelei Ma.  Integration of histopathological images and multi-dimensional omics analyses predicts molecular features and prognosis in high-grade serous ovarian cancer.  Gynecologic Oncology, 163, 2021.
  * [41] Kevin M. Boehm, Emily A. Aherne, Lora Ellenson, Ines Nikolovski, Mohammed Alghamdi, Ignacio Vázquez-García, Dmitriy Zamarin, Kara Long Roche, Ying Liu, Druv Patel, Andrew Aukerman, Arfath Pasha, Doori Rose, Pier Selenica, Pamela I. Causa Andrieu, Chris Fong, Marinela Capanu, Jorge S. Reis-Filho, Rami Vanguri, Harini Veeraraghavan, Natalie Gangai, Ramon Sosa, Samantha Leung, Andrew McPherson, Jian Jiong Gao, Yulia Lakhman, and Sohrab P. Shah.  Multimodal data integration using machine learning improves risk stratification of high-grade serous ovarian cancer.  Nature Cancer, 3:723–733, 6 2022.
  * [42] Jeffrey Boschman, Hossein Farahani, Amirali Darbandsari, Pouya Ahmadvand, Ashley Van Spankeren, David Farnell, Adrian B. Levine, Julia R. Naso, Andrew Churg, Steven J.M. Jones, Stephen Yip, Martin Köbel, David G. Huntsman, C. Blake Gilks, and Ali Bashashati.  The utility of color normalization for ai-based diagnosis of hematoxylin and eosin-stained pathology images.  Journal of Pathology, 256, 2022.
  * [43] Nicolas Elie, Florence Giffard, Cécile Blanc-Fournier, Pierre Marie Morice, Pierre Emmanuel Brachet, Soizic Dutoit, Benoît Plancoulaine, and Laurent Poulain.  Impact of automated methods for quantitative evaluation of immunostaining: Towards digital pathology.  Frontiers in Oncology, 12, 10 2022.
  * [44] Hossein Farahani, Jeffrey Boschman, David Farnell, Amirali Darbandsari, Allen Zhang, Pouya Ahmadvand, Steven J. M. Jones, David Huntsman, Martin Köbel, C. Blake Gilks, Naveena Singh, and Ali Bashashati.  Deep learning-based histotype diagnosis of ovarian carcinoma whole-slide pathology images.  Modern Pathology, 35:1983–1990, 12 2022.
  * [45] Yang Hu, Korsuk Sirinukunwattana, Kezia Gaitskell, Ruby Wood, Clare Verrill, and Jens Rittscher.  Predicting molecular traits from tissue morphology through self-interactive multi-instance learning.  In Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part II, pages 130–139. Springer, 2022\.
  * [46] Jun Jiang, Burak Tekin, Lin Yuan, Sebastian Armasu, Stacey J. Winham, Ellen L. Goode, Hongfang Liu, Yajue Huang, Ruifeng Guo, and Chen Wang.  Computational tumor stroma reaction evaluation led to novel prognosis-associated fibrosis and molecular signature discoveries in high-grade serous ovarian carcinoma.  Frontiers in Medicine, 9, 9 2022.
  * [47] Kokila R. Kasture, Dharmaveer Choudhari, and Pravin N. Matte.  Prediction and classification of ovarian cancer using enhanced deep convolutional neural network.  International Journal of Engineering Trends and Technology, 70:310–318, 3 2022.
  * [48] Piotr A Kowalski, Jakub Błoniarz, and Łukasz Chmura.  Convolutional neural networks in the ovarian cancer detection.  In Computational Intelligence and Mathematics for Tackling Complex Problems 2, pages 55–64. Springer, 2022.
  * [49] Tianling Liu, Ran Su, Changming Sun, Xiuting Li, and Leyi Wei.  Eocsa: Predicting prognosis of epithelial ovarian cancer with whole slide histopathological images.  Expert Systems with Applications, 206, 11 2022.
  * [50] Camilla Nero, Luca Boldrini, Jacopo Lenkowicz, Maria Teresa Giudice, Alessia Piermattei, Frediano Inzani, Tina Pasciuto, Angelo Minucci, Anna Fagotti, Gianfranco Zannoni, Vincenzo Valentini, and Giovanni Scambia.  Deep-learning to predict brca mutation and survival from digital h&e slides of epithelial ovarian cancer.  International Journal of Molecular Sciences, 23, 10 2022.
  * [51] Jennifer Salguero, Prateek Prasanna, Germán Corredor, Angel Cruz-Roa, David Becerra, and Eduardo Romero.  Selecting training samples for ovarian cancer classification via a semi-supervised clustering approach.  In Medical Imaging 2022: Digital and Computational Pathology, volume 12039, pages 20–24. SPIE, 2022.
  * [52] Ching Wei Wang, Cheng Chang Chang, Yu Ching Lee, Yi Jia Lin, Shih Chang Lo, Po Chao Hsu, Yi An Liou, Chih Hung Wang, and Tai Kuang Chao.  Weakly supervised deep learning for prediction of treatment effectiveness on ovarian cancer from histopathology images.  Computerized Medical Imaging and Graphics, 99, 7 2022.
  * [53] Chandra Holback, Rose Jarosz, Fred Prior, David G Mutch, Priya Bhosale, Kimberly Garcia, Yueh Lee, Shanah Kirk, Cheryl A Sadow, Seth Levine, Evis Sala, Pierre Elnajjar, Tara Morgan, and Brad Erickson.  The cancer genome atlas ovarian cancer collection (tcga-ov) (version 4) [data set].  The Cancer Imaging Archive, 2016.
  * [54] Martin Köbel, Steve E. Kalloger, Patricia M. Baker, Carol A. Ewanowich, Jocelyne Arseneau, Viktor Zherebitskiy, Soran Abdulkarim, Samuel Leung, Máire A. Duggan, Dan Fontaine, Robin Parker, David G. Huntsman, and C. Blake Gilks.  Diagnosis of ovarian carcinoma cell type is highly reproducible: A transcanadian study.  American Journal of Surgical Pathology, 34, 2010.
  * [55] Maximilian Ilse, Jakub Tomczak, and Max Welling.  Attention-based deep multiple instance learning.  In International conference on machine learning, pages 2127–2136. PMLR, 2018.
  * [56] Ming Y. Lu, Drew F.K. Williamson, Tiffany Y. Chen, Richard J. Chen, Matteo Barbieri, and Faisal Mahmood.  Data-efficient and weakly supervised computational pathology on whole-slide images.  Nature Biomedical Engineering, 5, 2021.
  * [57] Kelei He, Chen Gan, Zhuoyuan Li, Islem Rekik, Zihao Yin, Wen Ji, Yang Gao, Qian Wang, Junfeng Zhang, and Dinggang Shen.  Transformers in medical image analysis: A review.  Intelligent Medicine, 2022.

##  Appendix A Search strategy

Searches for all databases are shown here, with any text which is not directly
input to the search bar in bold font. These searches are each a combination of
three aspects - artificial intelligence, ovarian cancer, and histopathology.
No filters were applied, and all options were left on their default settings.

###  A.1 PubMed

(”Machine Learning”[Mesh] OR ”Artificial Intelligence”[Mesh] OR ”Neural
Networks, Computer”[Mesh] OR ”support vector machine”[MeSH] OR “Deep
Learning”[Mesh] OR ”diagnosis, computer-assisted”[Mesh] OR “Machine
learn$\ast$” OR “Artificial Intelligen$\ast$” OR (ML[Title/Abstract] NOT
($\mu$gml[Title/Abstract] OR $\mu$/ml[Title/Abstract] OR mgml[Title/Abstract]
OR pgml[Title/Abstract] OR ngml[Title/Abstract] OR uiml[Title/Abstract] OR
iuml[Title/Abstract] OR miuml[Title/Abstract] OR muiml[Title/Abstract] OR
uml[Title/Abstract] OR gml[Title/Abstract] OR mlkg[Title/Abstract] OR
milliliter$\ast$[Title/Abstract])) OR AI[Title/Abstract] OR “Computer Vision”
OR “Neural network$\ast$” OR “Deep Network$\ast$” OR “Computer-aided
Diagnosis” OR “Computer aided Diagnosis” OR Perceptron$\ast$ OR “Convolutional
Network$\ast$” OR “Recurrent Network$\ast$” OR “Graph Network$\ast$” OR “Deep
Learn$\ast$” OR “Deep-Learn$\ast$” OR Backprop$\ast$ OR “support vector$\ast$”
OR ensemble$\ast$ OR “random forest$\ast$” OR “nearest neighbor$\ast$” OR
“nearest neighbour$\ast$” OR “k-nearest neighbor$\ast$” OR “k-nearest
neighbour$\ast$” OR “Gradient boost$\ast$” OR “XGBoost$\ast$” OR
“segmentation” OR “instance learning” OR “multi-instance learning” OR “Active
Learning”)

AND ((ovar$\ast$ AND (cancer$\ast$ OR mass$\ast$ OR carcinoma$\ast$ OR
tumour$\ast$ OR tumor$\ast$ OR neoplasm$\ast$ OR malignan$\ast$ OR
”carcinoma”[Mesh] OR ”neoplasms”[Mesh])) OR “Ovarian Neoplasms”[Mesh])

AND ((digit$\ast$ AND patholog$\ast$) OR “tissue microarray$\ast$” OR
histopath$\ast$ OR histolog$\ast$ OR “Whole Slide Imag$\ast$” OR “Tissue
slide$\ast$” OR “pathology slide$\ast$” OR “pathology image$\ast$” OR
Immunohistochem$\ast$ OR ((Haematoxylin OR Hematoxylin) AND Eosin) OR
Histology[Mesh])

###  A.2 Scopus

TITLE-ABS-KEY(“Machine learn$\ast$” OR “Artificial Intelligen$\ast$” OR (“ML”
AND NOT “$\ast$ $\mu$ ml” AND NOT “$\ast$g ml” AND NOT “$\ast$ui ml” AND NOT
“$\ast$Ul ml” AND NOT “$\ast$iu ml” AND NOT “$\ast$u ml” AND NOT “$\ast$g ml”
AND NOT “$\ast$ml kg” AND NOT milliliter$\ast$) OR AI OR “Computer Vision” OR
“Neural network$\ast$” OR “Deep Network$\ast$” OR “Computer-aided Diagnosis”
OR “Computer aided Diagnosis” OR Perceptron$\ast$ OR “Convolutional
Network$\ast$” OR “Recurrent Network$\ast$” OR “Graph Network$\ast$” OR “Deep
Learn$\ast$” OR “Deep-Learn$\ast$” OR Backprop$\ast$ OR “support vector$\ast$”
OR ensemble$\ast$ OR “random forest$\ast$” OR “nearest neighbor$\ast$” OR
“nearest neighbour$\ast$” OR “k-nearest neighbor$\ast$” OR “k-nearest
neighbour$\ast$” OR “Gradient boost$\ast$” OR “XGBoost$\ast$” OR
“segmentation” OR “instance learning” OR “multi-instance learning” OR “Active
Learning”)

AND TITLE-ABS-KEY(ovar$\ast$ AND (cancer$\ast$ OR mass$\ast$ OR
carcinoma$\ast$ OR tumour$\ast$ OR tumor$\ast$ OR neoplasm$\ast$ OR
malignan$\ast$))

AND TITLE-ABS-KEY((digit$\ast$ AND patholog$\ast$) OR “tissue
microarray$\ast$” OR histopath$\ast$ OR histolog$\ast$ OR “Whole Slide
Imag$\ast$” OR “Tissue slide$\ast$” OR “pathology slide$\ast$” OR “pathology
image$\ast$” OR Immunohistochem$\ast$ OR ((Haematoxylin OR Hematoxylin) AND
Eosin))

###  A.3 Web of Science

(ALL=(“Machine learn$\ast$” OR “Artificial Intelligen$\ast$” OR “Computer
Vision” OR “Neural network$\ast$” OR “Deep Network$\ast$” OR “Computer-aided
Diagnosis” OR “Computer aided Diagnosis” OR Perceptron$\ast$ OR “Convolutional
Network$\ast$” OR “Recurrent Network$\ast$” OR “Graph Network$\ast$” OR “Deep
Learn$\ast$” OR “Deep-Learn$\ast$” OR Backprop$\ast$ OR “support vector$\ast$”
OR ensemble$\ast$ OR “random forest$\ast$” OR “nearest neighbor$\ast$” OR
“nearest neighbour$\ast$” OR “k-nearest neighbor$\ast$” OR “k-nearest
neighbour$\ast$” OR “Gradient boost$\ast$” OR “XGBoost$\ast$” OR
“segmentation” OR “instance learning” OR “multi-instance learning” OR “Active
Learning”) OR TS=(AI OR (“ML” NOT (“$\ast$ $\mu$ ml” OR “$\ast$g ml” OR
“$\ast$ui ml” OR “$\ast$Ul ml” OR “$\ast$iu ml” OR “$\ast$u ml” OR “$\ast$g
ml” OR “$\ast$ml kg” OR milliliter$\ast$))))

AND ALL=(ovar$\ast$ AND (cancer$\ast$ OR mass$\ast$ OR carcinoma$\ast$ OR
tumour$\ast$ OR tumor$\ast$ OR neoplasm$\ast$ OR malignan$\ast$))

AND ALL=((digit$\ast$ AND patholog$\ast$) OR “tissue microarray$\ast$” OR
histopath$\ast$ OR histolog$\ast$ OR “Whole Slide Imag$\ast$” OR “Tissue
slide$\ast$” OR “pathology slide$\ast$” OR “pathology image$\ast$” OR
Immunohistochem$\ast$ OR ((Haematoxylin OR Hematoxylin) AND Eosin))

###  A.4 Cochrane Central Register of Controlled Trials

Search #1:

All text: (“Machine learn$\ast$” OR “Artificial Intelligen$\ast$” OR “Computer
Vision” OR “Neural network$\ast$” OR “Deep Network$\ast$” OR “Computer-aided
Diagnosis” OR “Computer aided Diagnosis” OR Perceptron$\ast$ OR “Convolutional
Network$\ast$” OR “Recurrent Network$\ast$” OR “Graph Network$\ast$” OR “Deep
Learn$\ast$” OR “Deep-Learn$\ast$” OR Backprop$\ast$ OR “support vector$\ast$”
OR ensemble$\ast$ OR “random forest$\ast$” OR “nearest neighbor$\ast$” OR
“nearest neighbour$\ast$” OR “k-nearest neighbor$\ast$” OR “k-nearest
neighbour$\ast$” OR “Gradient boost$\ast$” OR “XGBoost$\ast$” OR
“segmentation” OR “instance learning” OR “multi-instance learning” OR “Active
Learning”)

Search #2:

Title-Abstract-Keyword: (“AI” OR (“ML” NOT (“$\ast$ $\mu$ ml” OR “$\ast$g ml”
OR “$\ast$ui ml” OR “$\ast$Ul ml” OR “$\ast$iu ml” OR “$\ast$u ml” OR “$\ast$g
ml” OR “$\ast$ml kg” OR milliliter$\ast$))) in Title Abstract Keyword

Search #3:

All text: (ovar$\ast$ AND (cancer$\ast$ OR mass$\ast$ OR carcinoma$\ast$ OR
tumour$\ast$ OR tumor$\ast$ OR neoplasm$\ast$ OR malignan$\ast$))

AND ((digit$\ast$ AND patholog$\ast$) OR “tissue microarray$\ast$” OR
histopath$\ast$ OR histolog$\ast$ OR “Whole Slide Imag$\ast$” OR “Tissue
slide$\ast$” OR “pathology slide$\ast$” OR “pathology image$\ast$” OR
Immunohistochem$\ast$ OR ((Haematoxylin OR Hematoxylin) AND Eosin))

Final search:

(#1 OR #2) AND #3

###  A.5 WHO-ICTRP

((“Machine learn$\ast$” OR “Artificial Intelligen$\ast$” OR “Computer Vision”
OR “Neural network$\ast$” OR “Deep Network$\ast$” OR “Computer-aided
Diagnosis” OR “Computer aided Diagnosis” OR Perceptron$\ast$ OR “Convolutional
Network$\ast$” OR “Recurrent Network$\ast$” OR “Graph Network$\ast$” OR “Deep
Learn$\ast$” OR “Deep-Learn$\ast$” OR Backprop$\ast$ OR “support vector$\ast$”
OR ensemble$\ast$ OR “random forest$\ast$” OR “nearest neighbor$\ast$” OR
“nearest neighbour$\ast$” OR “k-nearest neighbor$\ast$” OR “k-nearest
neighbour$\ast$” OR “Gradient boost$\ast$” OR “XGBoost$\ast$” OR
“segmentation” OR “instance learning” OR “multi-instance learning” OR “Active
Learning”) OR (“AI” OR (“ML” NOT (“$\mu$/ml” OR “g/ml” OR “ui/ml” OR “Ul/ml”
OR “iu/ml” OR “u/ml” OR “g/ml” OR “ml/kg” OR milliliter$\ast$))))

AND (ovar$\ast$ AND (cancer$\ast$ OR mass$\ast$ OR carcinoma$\ast$ OR
tumour$\ast$ OR tumor$\ast$ OR neoplasm$\ast$ OR malignan$\ast$))

AND ((digit$\ast$ AND patholog$\ast$) OR “tissue microarray$\ast$” OR
histopath$\ast$ OR histolog$\ast$ OR “Whole Slide Imag$\ast$” OR “Tissue
slide$\ast$” OR “pathology slide$\ast$” OR “pathology image$\ast$” OR
Immunohistochem$\ast$ OR ((Haematoxylin OR Hematoxylin) AND Eosin))

##  Appendix B Data extraction

gray!30 Category | Fields
---|---
Overview | Internal ID. Lead author. Year. Conference/Journal name.
Data |  Number of development images. Total number of images. Type of samples. FFPE/Frozen. Size of images. Tissue of origin. Number of development patients. Total number of patients. Number of data collection centres. Type of stain. Number of stainers. Scanners. Number of scanner types. Number of tissue processing centes. Data origin countries. Number of pathologists for data labelling. Online dataset. Prospective/retrospective. Clinical/research tissue. Data annotation. Maximum magnification available. Supplementary datatypes. Data exclusion reasons. Number of images excluded. Other cancer types included.
Methods |  Outcome. Outcome measure/classes. Outcome standards/definition. Magnifications used. Patch sizes. Patches per image. Task type. Feature extraction type. Feature extractors. AI in main method. Other AI methods. Optimiser. Number of external validations. Differences to external validation set. Total external validation images. Number of cross-validation folds. Number of non-novel methods compared. Number of GPUs. Type of GPUs.
Results |  Internal test accuracy, error bounds. AUC, error bounds. Sensitivity/specificity, error bounds. Other metric 1, error bounds. Other metric 2, error bounds. Other metric 3, error bounds. External training type. External test accuracy, error bounds. AUC, error bounds. Sensitivity/specificity, error bounds. Other metric 1, error bounds. Other metric 2, error bounds. Other metric 3, error bounds. Type of error bounds. Model training time. Visualisation type.
Miscellaneous | Code availability. Data availability. Notes
Table 4: Summary of the fields used for data extraction. The full form is
available at www.github.com/scjjb/OvCaReview.


We present data from related papers which observed a transition from deep convection to shallow convection as the depth ratio is altered. Each of the papers we consider uses a two-domain approach in modeling the system (so that the interface conditions can be explicitly enforced), and they each use a linear equation of state for the Boussinesq approximation. There are a variety of approaches taken in each work, from analytical methods to numerical simulations to experiments. Table <ref> references the approach taken in each paper, and we expand on the analytical approaches here: [Chen & Chen, 1988, McKay, 1998, Straughan, 2002, Hirata et al., 2007, Yin et al., 2013] each use linear stability theory, [Hill & Straughan, 2009, McCurdy et al., 2019] use nonlinear stability arguments,  and [Han et al., 2020] investigate this bifurcation using center manifold reduction theory.

Table <ref> shows the marked improvements in predicting the critical depth ratio $\dhatc$ using our theory from (<ref>) compared to predictions made using the old model. Despite several works operating under different assumptions about the system– like different governing equations (e.g., Darcy-Brinkman in lieu of Darcy), assuming the porous media has a high porosity, or using a non-Newtonian fluid– we find our new model produces an accurate prediction for the critical depth ratio.

Our formula in (<ref>) can be rearranged to solve for the critical $\epsilon_T$ or $\Da$ values, and two works, <cit.> and <cit.>, note how the transition from deep to shallow convection is affected by changing these parameters. Table <ref> shows our prediction for the critical values of these parameters alongside their true values. We once again find better agreement between the true values and predictions made with the new model compared to our old formula.

Paper                                                 c]@l@Gov.
Eqns                          Parameters                                                                              c]@l@“True” critical
depth ratio
from paper                   c]@l@Predicted
$\dhatc$ from
new model     c]@l@Predicted
$\dhatc$ from
old model
(analytical)                                   (a)    c]@l@$\sqrt{\Da}= 0.003$,
$\epsilon_T = 0.7$, $\alpha = 0.1$                     $\dhatc\in[0.12, 0.13]$                                                                                0.1291                                                                                0.1176
(experimental)                               –    c]@l@$\sqrt{\Da}= 0.002$,
$\epsilon_T = 0.7$, $\alpha = 0.1$                      $\dhatc\in[0.1, 0.11]$                                                                                0.1054                                                                                0.0960
(numerical)                                     (b)     c]@l@$\sqrt{\Da}= 0.889\times 10^{-5}$,
$\epsilon_T = 0.725$                      c]<EMAIL_ADDRESS>0.2]$,
with a “critical
value of $\dhatc=0.13$”     0.1287                                                                                0.1172
(analytical)                                      (a)     c]@l@$\Da=10^{-5}$, $\epsilon_T = 2.0$                                            $\dhatc\approx 0.2128$                                                                                0.2241                                                                                0.2041
(analytical)                                    (a)     c]@l@$\sqrt{\Da} = 3.279\times 10^{-3}$,
$\epsilon_T = 0.7$, $\alpha = 0.78$     $\dhatc\in[0.14, 0.15]$                                                                                0.1350                                                                                0.1229
(analytical)                                    (c)     c]@l@$\sqrt{\Da} = 0.003$,
$\epsilon_T = 0.7$, $\alpha = 0.1$     $\dhatc\in[0.12, 0.14]$                                                                                0.1226                                                                                0.1116
(analytical)                              (c)    c]@l@$\Da = 5\times 10^{-6},$
$\epsilon_T = 0.7$                                 $\dhatc\in[0.115, 0.117]$                                                                                0.1115                                                                                0.1015
(analytical)     (d)             c]@l@$\sqrt{\Da} = 0.002,$
$\epsilon_T=0.7,$ $\alpha=0.1$                         $\dhatc\in[0.1, 0.11]$                                                                                0.1054                                                                                0.0960
(d)    c]@l@$\sqrt{\Da} = 0.002,$
$\epsilon_T=0.6,$ $\alpha=0.1$                        $\dhatc\in[0.09, 0.10]$                                                                                0.0976                                                                                0.0889
(analytical)     (a)     c]@l@$\sqrt{\Da} = 5\times 10^{-3},$
$\epsilon_T = 0.7$, $\alpha = 1.0$          $\dhatc\approx .181$                                                                                0.1667                                                                                0.1518
(a)    c]@l@$\sqrt{\Da} = 1\times 10^{-3},$
$\epsilon_T = 0.7$, $\alpha = 1.0$          $\dhatc\approx 0.079$                                                                                0.0745                                                                                0.0679
(a)     c]@l@$\sqrt{\Da} = 5\times 10^{-3},$
$\epsilon_T = 0.5$, $\alpha = 1.0$          $\dhatc\approx 0.115$                                                                                0.1409                                                                                0.1283
(a)     c]@l@$\sqrt{\Da} = 5\times 10^{-3},$
$\epsilon_T = 1.0$, $\alpha = 1.0$           $\dhatc\approx 0.256$                                                                                0.2440                                                                                0.2222
(analytical)     (a)                             c]@l@$\Da = 25\times 10^{-6},$
$\epsilon_T = 0.7$, $\alpha = 1.0$                 $\dhatc\in[0.16, 0.17]$                                                                                0.1667                                                                                0.1518

Data from papers which note a critical depth ratio along with the prediction for the $\dhatc$ from our theory (<ref>) compared to the prediction made using the old model. The approach taken by the authors is noted alongside the citation, with a majority of the analytic approaches taking the form of investigating marginal stability curves and <cit.> making use of center manifold reduction theory to determine where the transition occurs.
Notation for the governing equations: (a) Navier-Stokes-Darcy-Boussinesq; (b) Navier-Stokes-Darcy-Brinkman-Boussinesq, with nonlinear Forchheimer term in Darcy; (c) Navier-Stokes-Darcy-Brinkman-Boussinesq;  (d) Navier-Stokes-Darcy-Boussinesq with an Oldroyd-B fluid. $\dagger$ from the two-domain approach in <cit.>.

1|l|Paper                                                 c]@l@Gov.
Eqns                          Parameters                                                                              c]@l@“True” critical
$\epsilon_T$ value
from paper                   c]@l@Predicted
$\epsilon_T^*$ from
new model     c]@l@Predicted
$\epsilon_T^*$ from
old model
(analytical)     (i)             c]@l@$\sqrt{\Da} = 0.002,$
$\hat{d}=0.1,$ $\alpha=0.1$                         $\epsilon_T^*\in[0.6, 0.7]$                                                                                0.6298                                                                                0.7595
(analytical)     (ii)                             c]@l@$\Da = 25\times 10^{-6},$
$\hat{d} = 0.2$, $\alpha = 1.0$                 $\epsilon_T^*\in[0.8, 0.9]$                                                                                1.0078                                                                                1.2151
1|l|Paper                                                 c]@l@Gov.
Eqns                          Parameters                                                                              c]@l@“True” critical
$\Da$ value
from paper                   c]@l@Predicted
$\Da^*$ from
new model     c]@l@Predicted
$\Da^*$ from
old model
(analytical)     (i)             c]@l@$\epsilon_T = 0.7,$
$\hat{d}=0.1,$ $\alpha=0.1$                         $\Da^*\approx 1.6\times10^{-5}$                                                                                $1.6395\times10^{-5} $                                                                                $2.3836\times10^{-5}$
(analytical)     (ii)                             c]@l@$\epsilon_T = 0.7,$
$\hat{d} = 0.2$, $\alpha = 1.0$                 $\Da^*\in\left(7\times 10^{-4}, 8\times 10^{-4}\right)$                                                                                $5.1815\times 10^{-5}$                                                                                $7.5333\times 10^{-5}$

Data from related papers which observed a transition from deep convection to shallow convection as the ratio of thermal diffusivities $\epsilon_T$ or the Darcy number $\Da$ is altered. We present our prediction for the critical value from the theory presented in (<ref>) (rearranged to solve for $\epsilon_T^*$ or $\Da^*$ instead of $\dhatc$) compared to the prediction made using the old model. The approach taken by the authors is noted alongside the citation, with <cit.> investigating marginal stability curves and <cit.> making use of center manifold reduction theory to determine where the transition occurs.
Notation for the governing equations: (i) Navier-Stokes-Darcy-Boussinesq with an Oldroyd-B fluid; (ii) Navier-Stokes-Darcy-Boussinesq.

natexlab#1#1#1#1 #1#1 #1#1#1#1#1#1
#1#1#1#1 #1#1 #1#1

[Al-Zamily, 2017]
Al-Zamily, A. M. J. 2017  Analysis of natural convection
and entropy generation in a cavity filled with multi-layers of porous medium
and nanofluid with a heat generation.  International Journal of Heat and
Mass Transfer  106,  1218–1231.

[Allen, 1984]
Allen, M. B. 1984 Collocation Techniques for Modeling
Compositional Flows in Oil Reservoirs.  Verlag Berlin Heidelberg:

[Allen & Khosravani, 1992]
Allen, M. B. & Khosravani, A. 1992  Solute transport
via alternating-direction collocation using the modified method of
characteristics.  Advances in water resources  15 (2),

[Bear, 1972]
Bear, J. 1972 Dynamics of Fluids in Porous Media.
New York, NY: Dover Publications Inc.

[Çeşmelioğlu & Rivière, 2008]
Çeşmelioğlu, A. & Rivière, B. 2008
Analysis of time-dependent Navier–Stokes flow coupled with Darcy
flow.  Journal of Numerical Mathematics  16 (4),  249–280.

[Çeşmelioğlu & Rivière, 2009]
Çeşmelioğlu, A. & Rivière, B. 2009
Primal discontinuous Galerkin methods for time-dependent coupled
surface and subsurface flow.  Journal of Scientific Computing
40 (1-3),  115–140.

[Chen & Chen, 1989]
Chen, Falin & Chen, CF 1989  Experimental
investigation of convective stability in a superposed fluid and porous layer
when heated from below.  Journal of Fluid Mechanics  207,

[Chen & Chen, 1988]
Chen, F. & Chen, C. F. 1988  Onset of finger
convection in a horizontal porous layer underlying a fluid layer.
Journal of Heat Transfer  110,  403–409.

[Chen & Chen, 1992]
Chen, F. & Chen, C. F. 1992  Convection in superposed
fluid and porous layers.  Journal of Fluid Mechanics  234,

[Chen et al., 2020]
Chen, W., Han, D., Wang, X. & Zhang, Y. 2020
Uniquely solvable and energy stable decoupled numerical schemes for the
Cahn–Hilliard–Navier–Stokes–Darcy–Boussinesq system.
Journal of Scientific Computing  85 (2),  1–28.

[Chen et al., 2021]
Chen, W., Han, D., Wang, X. & Zhang, Y. 2021
Conservative unconditionally stable decoupled numerical schemes for the
Cahn–Hilliard–Navier–Stokes–Darcy–Boussinesq system.
Numerical Methods for Partial Differential Equations .

[Chidyagwai & Rivière, 2007]
Chidyagwai, P. & Rivière, B. 2007  A weak
solution and a multinumerics solution of the coupled Navier–Stokes and
Darcy equations.  IMA Journal of Numerical Analysis .

[Chiu et al., 2020]
Chiu, S-H., Moore, M. N. J. & Quaife, B. 2020
Viscous transport in eroding porous media.  Journal of Fluid
Mechanics  893.

[Choblet et al., 2017]
Choblet, G., Tobie, G., Sotin, C.,
Běhounková, M., Čadek, O., Postberg, F. &
Souček, O. 2017  Powering prolonged hydrothermal
activity inside Enceladus.  Nature Astronomy  1 (12),

[Couston, 2021]
Couston, L-A. 2021  Turbulent convection in subglacial
lakes.  Journal of Fluid Mechanics  915.

[Curran & Allen, 1990]
Curran, M. C. & Allen, M. B. 1990  Parallel computing
for solute transport models via alternating direction collocation.
Advances in water resources  13 (2),  70–75.

[Discacciati & Quarteroni, 2009]
Discacciati, M. & Quarteroni, A. 2009
Navier-Stokes/Darcy coupling: modeling, analysis, and numerical
approximation.  Revista Matemàtica Complutense .

[Eastham et al., 2020]
Eastham, P. S., Moore, M. N. J., Cogan, N. G., Wang,
Q. & Steinbock, O. 2020  Multiphase modelling of
precipitation-induced membrane formation.  Journal of Fluid Mechanics

[Ewing, 1997]
Ewing, R. E. 1997  The need for multidisciplinary
involvement in groundwater contaminant simulations.  In Proceedings
of the Next Generation Environmental Models and Computational Methods,
pp. 227–245.  Philadelphia, PA: SIAM.

[Girault & Rivière, 2009]
Girault, V. & Rivière, B. 2009  DG
approximation of coupled Navier–Stokes and Darcy equations by
Beaver–Joseph–Saffman interface condition.  SIAM Journal on
Numerical Analysis  47 (3),  2052–2089.

[Han et al., 2020]
Han, D., Wang, Q. & Wang, X. 2020  Dynamic
transitions and bifurcations for thermal convection in the superposed free
flow and porous media.  Physica D: Nonlinear Phenomena  p. 132687.

[Han & Wang, 2016]
Han, D. & Wang, X. 2016  Decoupled energy-law
preserving numerical schemes for the Cahn-Hilliard-Darcy system.
Numerical Methods for Partial Differential Equations  32 (3),

[Hecht, 2012]
Hecht, F. 2012  New development in FreeFem++.  J.
Numer. Math.  20 (3-4),  251–265.

[Hill & Straughan, 2009]
Hill, A. & Straughan, B. 2009  Global stability for
thermal convection in a fluid overlying a highly porous material.
Proceedings of the Royal Society A  465,  207–217.

[Hirata et al., 2007]
Hirata, SC, Goyeau, B, Gobin, D, Carr, M & Cotta,
RM 2007  Linear stability of natural convection in superposed
fluid and porous layers: influence of the interfacial modelling.
International journal of heat and mass transfer  50 (7-8),

[Hirata et al., 2012]
Hirata, S. C., Goyeau, B. & Gobin, D. 2012
Onset of convective instabilities in under-ice melt ponds.  Physical
Review E  85 (6),  066306.

[Huppert & Neufeld, 2014]
Huppert, H. E. & Neufeld, J. A. 2014  The fluid
mechanics of carbon dioxide sequestration.  Annual review of fluid
mechanics  46,  255–272.

[Jones, 1973]
Jones, I. P. 1973  Low reynolds number flow past a porous
spherical shell.  Proceedings of the Cambridge Philosophical Society
73,  231–238.

[Khazmutdinova et al., 2019]
Khazmutdinova, K., Nof, D., Tremaine, D., Ye, M. &
Moore, M. N. J. 2019  A minimal model for predicting
ventilation rates of subterranean caves.  Journal of Cave and Karst
Studies  81 (4),  264–275.

[Lasser et al., 2021]
Lasser, J., Ernst, M. & Goehring, L. 2021
Stability and dynamics of convection in dry salt lakes.  Journal of
Fluid Mechanics  917.

[Le Bars & Worster, 2006]
Le Bars, M. & Worster, M. G. 2006a
Interfacial conditions between a pure fluid and a porous medium:
implications for binary alloy solidification.  Journal of Fluid
Mechanics  550,  149–173.

[Le Bars & Worster, 2006]
Le Bars, M. & Worster, M. G. 2006b
Solidification of a binary alloy: Finite-element, single-domain
simulation and new benchmark solutions.  Journal of Computational
Physics  216,  247–263.

[Le Reun & Hewitt, 2020]
Le Reun, T. & Hewitt, D. R. 2020  Internally heated
porous convection: An idealized model for Enceladus' hydrothermal
activity.  Journal of Geophysical Research: Planets  125 (7),

[Le Reun & Hewitt, 2021]
Le Reun, T. & Hewitt, D. R. 2021
High-Rayleigh-number convection in porous–fluid layers.  Journal
of Fluid Mechanics  920.

[Lyu & Wang, 2021]
Lyu, W. & Wang, X. 2021  Stokes–Darcy system,
small-Darcy-number behaviour and related interfacial conditions.
Journal of Fluid Mechanics  922.

[Mac Huang & Moore, 2022]
Mac Huang, J. & Moore, M. N. J. 2022  Morphological
attractors in natural convective dissolution.  Physical Review Letters
128 (2),  024501.

[Mac Huang et al., 2020]
Mac Huang, J., Tong, J., Shelley, M. & Ristroph,
L. 2020  Ultra-sharp pinnacles sculpted by natural convective
dissolution.  Proceedings of the National Academy of Sciences
117 (38),  23339–23344.

[Mac Huang et al., 2018]
Mac Huang, J., Zhong, J-Q., Zhang, J. & Mertz, L.
2018  Stochastic dynamics of fluid–structure interaction in
turbulent thermal convection.  Journal of Fluid Mechanics  854.

[McCurdy et al., 2019]
McCurdy, M., Moore, M. N. J. & Wang, X. 2019
Convection in a coupled free flow-porous media system.  SIAM Journal
on Applied Mathematics  79 (6),  2313–2339.

[McKay, 1998]
McKay, G 1998  Onset of bouyancy-driven convection in
superposed reacting fluid and porous layers.  Journal of engineering
mathematics  33 (1),  31–47.

[Moore, 2017]
Moore, M. N. J. 2017  Riemann-Hilbert problems for the
shapes formed by bodies dissolving, melting, and eroding in fluid flows.
Communications on Pure and Applied Mathematics  70 (9),

[Moore & Shelley, 2012]
Moore, M. N. J. & Shelley, M. J. 2012  A
weak-coupling expansion for viscoelastic fluids applied to dynamic settling
of a body.  Journal of Non-Newtonian Fluid Mechanics  183,

[Nield & Bejan, 2017]
Nield, D. & Bejan, A. 2017 Convection in Porous
Media, 5th edn.  Cham, Switzerland: Springer International

[Quaife & Moore, 2018]
Quaife, B. D. & Moore, M. N. J. 2018  A
boundary-integral framework to simulate viscous erosion of a porous medium.
Journal of Computational Physics  375,  1–21.

[Saffman, 1971]
Saffman, P. 1971  On the boundary condition at the
interface of a porous medium.  Stud. Appl. Math.  1,

[Sorribes et al., 2019]
Sorribes, I. C., Moore, M. N. J., Byrne, H. M. & Jain,
H. V. 2019  A biomechanical model of tumor-induced intracranial
pressure and edema in brain tissue.  Biophysical journal
116 (8),  1560–1574.

[Straughan, 2002]
Straughan, B. 2002  Effect of property variation and
modelling on convection in a fluid overlying a porous layer.
International Journal for Numerical and Analytical Methods in
Geomechanics  26,  75–97.

[Tatsuo et al., 1986]
Tatsuo, N., Toru, T., Mitsuhiro, S., Yuji, K. &
Hiroyuki, O. 1986  Numerical analysis of natural convection in
a rectangular enclosure horizontally divided into fluid and porous regions.
International Journal of Heat and Mass Transfer  29 (6),

[Temam, 1968]
Temam, R. 1968  Une méthode d'approximation de la
solution des équations de Navier-Stokes.  Bulletin de la
Société Mathématique de France  96,  115–152.

[Temam, 1969]
Temam, R. 1969a  Sur l'approximation
de la solution des équations de Navier-Stokes par la méthode des
pas fractionnaires (i).  Archive for Rational Mechanics and Analysis
32 (2),  135–153.

[Temam, 1969]
Temam, R. 1969b  Sur l'approximation
de la solution des équations de Navier-Stokes par la méthode des
pas fractionnaires (ii).  Archive for rational mechanics and analysis
33 (5),  377–385.

[Tyvand, 2002]
Tyvand, P. A. 2002  Onset of Rayleigh-Bénard
Convection in Porous Bodies.  In Transport Phenomena in
Porous Media II (ed. D. B. Ingham & I. Pop),  pp. 82–112.
Oxford: Pergamon.

[Valencia-Lopez & Ochoa-Tapia, 2001]
Valencia-Lopez, J. J. & Ochoa-Tapia, J. A. 2001  A
study of buoyancy-driven flow in a confined fluid overlying a porous layer.
International Journal of Heat and Mass Transfer  44 (24),

[Vilella et al., 2020]
Vilella, K., Choblet, G., Tsao, W-E. & Deschamps, F.
2020  Tidally heated convection and the occurrence of melting in icy
satellites: application to Europa.  Journal of Geophysical Research:
Planets  125 (3),  e2019JE006248.

[Wang & Wu, 2021]
Wang, X. & Wu, H. 2021  Global weak solutions to the
Navier–Stokes–Darcy–Boussinesq system for thermal convection in
coupled free and porous media flows.  Advances in Differential
Equations  26 (1/2),  1–44.

[Wykes et al., 2018]
Wykes, M. S. D., Mac Huang, J., Hajjar, G. A. &
Ristroph, L. 2018  Self-sculpting of a dissolvable body due to
gravitational convection.  Physical Review Fluids  3 (4),

[Yin et al., 2013]
Yin, Chen, Fu, Ceji & Tan, Wenchang 2013
Stability of thermal convection in a fluid-porous system saturated with
an Oldroyd-B fluid heated from below.  Transport in porous media
99 (2),  327–347.

[Yu et al., 2010]
Yu, S-H., Lee, K-S. & Yook, S-J. 2010  Natural
convection around a radial heat sink.  International Journal of Heat and
Mass Transfer  53 (13-14),  2935–2938.

[Yu et al., 2011]
Yu, S-H., Lee, K-S. & Yook, S-J. 2011  Optimum
design of a radial heat sink under natural convection.  International
Journal of Heat and Mass Transfer  54 (11-12),  2499–2505.

[Zhang & Libchaber, 2000]
Zhang, J. & Libchaber, A. 2000  Periodic boundary
motion in thermal turbulence.  Physical Review Letters
84 (19),  4361.

[Zhang et al., 2020]
Zhang, Y., Shan, L. & Hou, Y. 2020
Well-posedness and Finite Element approximation for the convection
model in superposed fluid and porous layers.  SIAM Journal on Numerical
Analysis  58 (1),  541–564.