# Machine learning approaches for COVID-19 detection from chest X-ray imaging:
A Systematic Review

Harold Brayan Arteaga-Arteaga  Department of Electronics and Automation,
Universidad Autónoma de Manizales, Manizales, Colombia  Melissa delaPava
Department of Electronics and Automation, Universidad Autónoma de Manizales,
Manizales, Colombia  Alejandro Mora-Rubio  Department of Electronics and
Automation, Universidad Autónoma de Manizales, Manizales, Colombia  Mario
Alejandro Bravo-Ortíz  Department of Electronics and Automation, Universidad
Autónoma de Manizales, Manizales, Colombia  Jesus Alejandro Alzate-Grisales
Department of Electronics and Automation, Universidad Autónoma de Manizales,
Manizales, Colombia  Daniel Arias-Garzón  Department of Electronics and
Automation, Universidad Autónoma de Manizales, Manizales, Colombia  Luis
Humberto López-Murillo  Department of Chemical Engineering, Universidad
Nacional de Colombia, Manizales, Colombia  Felipe Buitrago-Carmona  Department
of Computer Science, Universidad Autónoma de Manizales, Manizales, Colombia
Juan Pablo Villa-Pulgarín  Department of Electronics and Automation,
Universidad Autónoma de Manizales, Manizales, Colombia  Esteban Mercado-Ruiz
Department of Electronics and Automation, Universidad Autónoma de Manizales,
Manizales, Colombia  Simon Orozco-Arias  Department of Computer Science,
Universidad Autónoma de Manizales, Manizales, Colombia  Department of Systems
and informatics, Universidad de Caldas, Manizales, Colombia  M. Hassaballah
Faculty of Computers and Information, South Valley University, Qena, Egypt
Maria de la Iglesia-Vaya  Unidad Mixta de Imagen Biomédica FISABIO-CIPF,
Fundación para el Fomento de la Investigación Sanitario y Biomédica de la
Comunidad Valenciana, Valencia, Spain  Oscar Cardona-Morales  Department of
Electronics and Automation, Universidad Autónoma de Manizales, Manizales,
Colombia  Reinel Tabares-Soto  Department of Electronics and Automation,
Universidad Autónoma de Manizales, Manizales, Colombia

###### Abstract

There is a necessity to develop affordable, and reliable diagnostic tools,
which allow containing the COVID-19 spreading. Machine Learning (ML)
algorithms have been proposed to design support decision-making systems to
assess chest X-ray images, which have proven to be useful to detect and
evaluate disease progression. Many research articles are published around this
subject, which makes it difficult to identify the best approaches for future
work. This paper presents a systematic review of ML applied to COVID-19
detection using chest X-ray images, aiming to offer a baseline for researchers
in terms of methods, architectures, databases, and current limitations.

_Keywords_ COVID-19 $\cdot$ X-ray images $\cdot$ automatic detection $\cdot$
artificial intelligence $\cdot$ machine learning

##  1 Introduction

The first confirmed cases of COVID-19 disease appeared in Wuhan, Hubei
province, China back in December 2019 (Wang et al., 2020). The disease is
caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)
(Yuen et al., 2020), which is transmitted primarily through droplets of saliva
or discharge from the nose (World Health Organization, 2020a). It has spread
all over the world, and it was declared as a pandemic by the World Health
Organization (WHO) in March 2020 (World Health Organization, 2020b). Based on
data from Johns Hopkins University, as of October 13th, 2021, there have been
239,038,163 confirmed cases of COVID-19 around the world including 4,871,163
deaths (World Health Organization, 2021). According to the WHO, most infected
people develop a moderate illness with symptoms such as fever, dry cough, and
fatigue. In severe cases where patients need hospitalization, the symptoms
include breathing difficulties, chest pain, and loss of speech or movement
(World Health Organization, 2020a).

COVID-19 can be diagnosed using tools based on the detection of viral gene,
human antibody, or viral antigen, which requires qualified personnel and a
specialized laboratory. As a complementary diagnostic tool, doctors employ
medical imaging techniques such as chest X-ray or chest Computerized
Tomography (CT). The produced images offer information about the lungs and can
help radiologists to detect diseases like pneumonia, tuberculosis,
interstitial lung disease, pneumothorax, early lung cancer, among others (Anis
et al., 2020). These images also have proven to be effective for COVID-19
detection, as well as giving information about disease progression through the
evaluation of radiological findings (Ng et al., 2020).

  * •

Real-Time Reverse Transcription Polymerase Chain Reaction: the viral gene
detection by Real-Time Reverse Transcription Polymerase Chain Reaction (RT-
PCR) is very sensitive because it can detect a copy of a specific genomic
sequence, which has lead to the development of many commercial technologies
that use nasal or nasopharyngeal swabs along with RT-PCR for COVID-19
detection (Yüce et al., 2021).

However, the detection of COVID-19 using RT-PCR is complex, the materials are
sometimes slow to deliver, and the complete process can only be performed by
qualified clinical laboratory personnel, which take over 24 hours from taking
the sample to getting the analysis results. It is also expensive due to one
kit can cost over 100 USD and setting a lab costs more than 15,000 USD.
Additionally, many factors like storage, collection, processing, and genomic
mutations, can lead to incorrect results (Afzal, 2020). Other drawbacks
include low availability in some countries (Aziz et al., 2020), and high
false-negative rates (Fan et al., 2020).

  * •

Computed Tomography scan: this is an advanced technique that allows to
generates detailed 3D images of organs and soft tissues (Ohata et al., 2021).
Unlike RT-PCR, a CT scan is fast to obtain and it is relatively easy to
perform. It has been recently reported that this technique shows typical
features of COVID-19 like ground-glass opacities and multifocal patchy
consolidation, even in patients with negative PCR but clinical symptoms (Ai et
al., 2020). However, decontaminate CT equipment after scanning COVID-19
patients may damage it. Thus, to minimize the risk of cross-infection, it is
suggested to use portable devices like chest radiography, which is already a
triage tool in many hospitals Wong et al. (2020).

  * •

Chest X-ray: X-ray refers to a medical imaging technique that uses radiation
to generate an image of internal structures of the human body. The main
elements that are assessed using X-ray are bones, which appear white on the
image; soft tissues, which appear as light gray; fat, which appears gray; and
gas, which appears black (Anis et al., 2020). In particular, a chest X-ray
image allows a doctor to evaluate multiple organs, structures and conditions
Breiding (2009). It is one of the most used methods to diagnose pneumonia
worldwide (Jaiswal et al., 2019).

Chest X-ray devices can be portable, are affordable, fast, and gives the
patient a lower radiation dose than CT. It has been reported that common CT
findings can also be detected on chest X-ray images, even in patients with
initial negative RT-PCR for COVID-19. However, the diagnosis of COVID-19 using
chest X-ray images is more difficult than using CT or other imaging modalities
and can only be performed by specialist physicians, which scarce (Narin et
al., 2021; Wong et al., 2020).

There is a trade-off between quality and accessibility when choosing the
imaging technique to use. CT produces a higher quality image but requires a
much more complex device, not always available in many institutions. On the
contrary, X-ray devices are much more affordable, can be portable, and are
less harmful, given that a single CT scan can deliver a median effective
radiation dose as high as 442 chest X-ray series (Breiding, 2009).

Concerning the radiological findings on chest X-ray and CT associated with
COVID-19 pneumonia, the most common are ground glass opacities. These marks
are usually bilateral, meaning they affect both lungs and are more likely to
be located in the periphery and lower areas of the lungs (Kaufman et al.,
2020; Yasin and Gouda, 2020). They can be seen in chest X-ray images, or CT
images as regions of increased whiteness due to the augmented density
(Cleverley et al., 2020), which do not cover blood vessels and airway walls
completely (Rousan et al., 2020). As the disease progresses, this finding
becomes denser and covers blood vessels and airway walls on the image,
becoming consolidations. Fig. 1 presents a comparison of the chest X-ray
images for a COVID-19 negative subject and a COVID-19 positive subject;
additionally, for the COVID-19 case, the image on the right shows the masks
over the regions of ground glass opacities (yellow) and consolidations
(purple).

Figure 1: Examples of chest X-ray images for a COVID-19 negative subject, a
COVID-19 positive subject. The last one shows the masks over the regions of
ground glass opacities (yellow) and consolidations (purple) on the region of
interest.

The WHO solidarity consortium from February 2021 presents a study to find a
drug against COVID-19, however, it was found that the mortality, initiation of
ventilation, and hospitalization duration were not definitely reduced by any
trial drug. Until now, no specific drug has been found against COVID-19 WHO
Solidarity Trial Consortium (2021). Currently, there are 153 vaccine
candidates, 476 vaccine trials ongoing, 23 vaccines approved by at least one
country, and 7 vaccines approved for use by the WHO against COVID-19. The
AstraZeneca vaccine is the one approved in the largest number of countries,
followed by Pfizer/BioNTech, Moderna, and Janssen (WHO, 2021).

Artificial Intelligence (AI) techniques, including Machine Learning (ML), can
be used for COVID-19 diagnosis from chest X-ray images and set foundations for
automatic decision-making support systems (Arteaga-Arteaga et al., 2022). AI
refers to the process of providing computer features from human intelligence.
ML is a subset of AI that holds the mathematical models used to achieve this
task, whereas Deep Learning (DL) is a subset of ML itself and relates the
models and algorithms based on neural networks (Goodfellow et al., 2016). In
general, ML and DL techniques are designed to extract features and find
relationships between data samples. Thereby, these approaches are well-suited
for tasks relying on the human experience (Orozco-Arias et al., 2019; Tabares-
Soto et al., 2019; Bravo Ortíz et al., 2021; Reinel et al., 2021; Arteaga-
Arteaga et al., 2021) such as classifying a chest X-ray image as positive or
negative for COVID-19. Besides decision-making support systems in the medicine
and healthcare field, AI has been used to perform tasks from managing medical
data and analyzing health plans to drug development and health monitoring
(Amisha et al., 2019). AI applications in medicine aim to improve diagnostic
performance and offer a better quality of service (Ahuja, 2019).

Regarding the detection of COVID-19 in chest X-ray images using AI techniques,
most research papers propose a transfer learning approach using Convolutional
Neural Networks (CNNs) such as VGG19, Inception, and MobileNet (Pham, 2021). A
different approach creates novel CNNs to classify chest X-ray images as
positive or negative for COVID-19 (Hussain et al., 2021). Different
traditional ML approaches have been proposed involving a manual feature
extraction stage employing texture or morphological descriptors of the images
(Hussain et al., 2020; Pereira et al., 2020). The availability of data is
probably the biggest limitation when designing AI systems to detect COVID-19,
although nowadays there are several public image databases, the quality of
images and information is highly variable, which makes it difficult for
researchers to evaluate their systems on appropriate conditions. Furthermore,
there is not a standard benchmark to evaluate and compare the different
proposals, which in combination with data variability, makes the reported
results difficult to compare with each other.

The paper is organized as follows: Section Survey Methodology explains the
criteria used to perform the literature review; Section Development Of The
Subject presents the results and the state of the art; and Section Conclusions
and Future Work.

##  2 Survey Methodology

A systematic review of scientific papers was conducted, which explains the
design and implementation of CNN, ML algorithms, or segmentation methods, for
COVID-19 classification from chest X-ray images.

###  2.1 Identification of the need for a review

Given the need to develop more efficient and effective diagnostics tools for
the COVID-19 disease, many research papers have been written since the
beginning of the pandemic. One estimate suggests that more than 200,000
research papers have been published in journals and preprints repositories
only in 2020 (Else, 2020). Therefore there is a need for a new state-of-the-
art review.

State-of-the-art works up to March 21, 2021, are summarized in Table 1, where
is shown if they specify the corresponding preprocessing techniques and
datasets used in the selected works, the date of search, whether or not it is
systematic, the number of search databases used, the data modalities included
in the work, the number of X-ray related articles analyzed and the number of
X-ray related databases described. Table 1 shows that some of the available
state-of-the-art works do not follow a systematic approach, and only include
works up to July 2020 or before, unlike our work that covers from January 1,
2020, to March 21, 2021. Additionally, our work uses the largest number of
search databases and some of the available state-of-the-art include a limited
amount of papers that identify COVID-19 using X-ray images.

The description of the available datasets presented in this work also fills a
gap in the literature, since the available works barely describe some of them.
We also present the preprocessing techniques and the specific datasets used on
each work selected for this literature review, what is not included in any of
the available works, along with the models, tasks and results described help
researchers to build a complete panorama of the actual strategies used for the
detection of COVID-19 using X-ray images. Our work is a complement to many of
the available works since they invest more effort in discussing the risk of
bias, recommendations, and deficiencies than in the specific methodologies
details as we do.

Therefore, our bibliographic review contributes relevant and up-to-date
information about the development of AI-based systems to detect COVID-19 from
chest X-ray images. It will also offer a baseline for researchers regarding
methods, architectures, databases, and current limitations.

###  2.2 Research questions

In order to describe the state-of-the-art approaches for COVID-19 detection,
this paper aims to answer the following questions:

  * •

Which are the different architectures and novel components of CNNs used to
detect COVID-19 on chest X-ray images?

  * •

What are the detection performances of COVID-19 on chest X-ray images using
CNNs?

  * •

Which digital image databases are the most used for COVID-19 detection on
chest X-ray images?

  * •

Which segmentation methods are applied on chest X-ray images for the automatic
detection of COVID-19?

###  2.3 Bibliographic search

The key words chosen for this search are:

  * •

COVID-19.

  * •

Deep Learning.

  * •

Machine Learning.

  * •

Classification.

  * •

Segmentation.

  * •

Chest X-ray.

After defining the search terms, the search string was built with logical
operators. Due to COVID-19 being a disease that emerged in late 2019, the
search is limited between 2020 - Present, only in the English language. The
general search string is: Covid-19 AND (Machine Learning OR Deep Learning) AND
(Classification OR Segmentation) AND X-ray. Table 2 shows the databases and
search strings used for the review. The gray literature search included papers
with novel COVID-19 classification methods from chest X-ray images.

Table 1: Related works

Article |  Database specification |  Preprocessing specification |  Date of search |  Date of publication |  Systematic |  Search databases |  Articles analyzed |  Databases described
---|---|---|---|---|---|---|---|---
Rahman et al. (2021) |  ✓ |  ✗ |  - |  March 02, 2021 |  ✗ |  - |  23 |  12
Nayak et al. (2021a) |  ✗ |  ✗ |  - |  March 20, 2021 |  ✓ |  4 |  41 |  1
(Wynants et al., 2020) |  ✗ |  ✗ |  July 01, 2020 |  April 07, 2021 |  ✓ |  5 |  22 |  -
Shi et al. (2021) |  ✗ |  ✗ |  March 31, 2020 |  April 16, 2020 |  ✗ |  - |  4 |  -
Swapnarekha et al. (2020) |  ✗ |  ✗ |  May 03, 2020 |  May 26, 2020 |  ✗ |  - |  12 |  -
Waleed Salehi et al. (2020) |  ✗ |  ✗ |  - |  June 23, 2020 |  ✗ |  - |  5 |  -
Albahri et al. (2020) |  ✗ |  ✗ |  May 15, 2020 |  June 25, 2020 |  ✓ |  4 |  11 |  -
Bansal et al. (2020) |  ✗ |  ✗ |  - |  August 01, 2020 |  ✗ |  - |  - |  -
Syeda et al. (2021) |  ✗ |  ✗ |  June 27, 2020 |  January 11, 2021 |  ✓ |  3 |  22 |  23
Roberts et al. (2021) |  ✗ |  ✗ |  - |  March 15, 2021 |  ✓ |  - |  22 |  -
Our work |  ✓ |  ✓ |  March 21, 2021 |  - |  ✓ |  6 |  23 |  18

Table 2: Databases and search strings for literature review | Name of the
---
search database
Search string
Scopus |  | TITLE-ABS-KEY (Covid-19 AND (Machine Learning OR Deep Learning)
---
AND (Classification OR Segmentation) AND X-ray)
Web of Science |  | (SUBJECT OR TITLE: Covid-19 AND (Machine Learning OR Deep Learning)
---
AND (Classification OR Segmentation) AND X-ray)
SpringerLink |  | Covid-19 AND (Machine Learning OR Deep Learning)
---
AND (Classification OR Segmentation) AND X-ray.
PubMed |  | Covid-19 AND (Machine Learning OR Deep Learning)
---
AND (Classification OR Segmentation) AND X-ray.
IEEE Xplore |  | Covid-19 AND (Machine Learning OR Deep Learning)
---
AND (Classification OR Segmentation) AND X-ray
Google Scholar |  | Covid-19 AND (Machine Learning OR Deep Learning)
---
AND (Classification OR Segmentation) AND X-ray.

###  2.4 Inclusion and exclusion criteria

The inclusion criteria taken into account are:

  * •

Papers published in Journals.

  * •

Papers written in English.

  * •

Papers found in the databases in Table 2.

  * •

Papers that use DL o ML to detect COVID-19 from chest X-ray images.

  * •

Papers using novel methods for COVID-19 detection.

The exclusion criteria taken into account are:

  * •

COVID-19 classification or segmentation papers without application of DL or ML
methods.

  * •

COVID-19 classification papers that do not use chest X-ray.

  * •

Papers with methods that include X-ray and CT simultaneously in the methods
training.

###  2.5 Data extraction and synthesis

We conduct a systematic literature review by applying the preferred reporting
items for systematic reviews and meta-analyses guidelines (the PRIMA
statement), the Fig. 2 presents the number of articles obtained on each step
of the guidelines (Moher, 2009).

Figure 2: PRISMA flow diagram. PRISMA flow chart for search and article
screening process. From: (Moher, 2009)

Based on the search string provided in Table 2, approximately 720 papers are
found and 120 papers per database are analyzed, from which 20 relevant papers
per database are pre-selected. The pre-selection is done using the number of
citations and the novelty of the COVID-19 detection method proposed. It is
relevant to clarify that we found 14 repeated papers that are rejected;
therefore, the pre-selection of the papers leads to 106 documents. The
remaining 106 articles are filtered according to:

  * •

Title: After reading the title, 40 are accepted and 66 papers are rejected.

  * •

Abstract: After reading the abstract, 28 are accepted and 12 papers are
rejected.

  * •

Full text: After reading the entire text, 23 papers are accepted and 5 papers
are rejected.

Figure 3 presents the percentage distribution of the selected papers in the
databases.

Figure 3: Percentage of papers selected for bibliographic review in different
databases.

##  3 Development of the subject

As mentioned in the previous section, a total of 23 research papers are
selected for the systematic review. This paper aims to cover novel approaches
and offer a general overview of how AI has been applied to COVID-19 diagnosis.

###  3.1 X-ray images databases

Table LABEL:table:databases presents the most relevant databases used in the
selected articles, they are described along with the total number of images,
and the classes. This table includes not only databases with COVID-19 samples
but also other chest diseases, due to some of the works published regarding
the detection of COVID-19, which intend to classify multiple chest conditions.
The most often classified illnesses along with COVID-19 are pneumonia viral,
pneumonia bacterial, and tuberculosis. The COVID-19 Image Data Collection
published by Paul Cohen et al. (2020) is the most used database, it was one of
the first ones to be released and more images are included over time. It also
provides prospective metadata like survival, ICU stay, intubation events,
blood tests, location, and freeform clinical notes. The most regularly used
databases in this regard with X-ray samples of other chest diseases are
ChestX-ray8 (CRX8), CheXpert, Chest X-ray Images (Pneumonia), and Tuberculosis
chest X-ray.

Table 3: Image databases for COVID-19 research Item |  Dataset name |  # Images |  Classes
---|---|---|---
A |  HM Hospitales (HM Hospitals, 2020) |  5,560 |  COVID-19
B |  BIMCV-COVID19 (Vayá et al., 2021) |  3,013 |  COVID-19
C |  Actualmed COVID-19 chest X-rays (Actualmed et al., 2020) |  188 |  COVID-19; Normal
D |  ChinaSet - The ShenzhenSet (Jaeger et al., 2014) |  662 |  Pneumonia; Normal
E |  Montgomery (Jaeger et al., 2014) |  138 |  Pneumonia; Normal
F |  ChestX-ray8 (CRX8) (Wang et al., 2017) |  61,790 |  Pneumonia; Normal
G |  CheXpert (Irvin et al., 2019) |  4,623 |  Pneumonia
H |  MIMIC-CXR (Johnson et al., 2019) |  16,399 |  Pneumonia
I |  COVID-19 Image Data Collection (Paul Cohen et al., 2020) |  481 |  Viral (COVID-19; SARS; MERS; among others); Bacterial; Others
J |  Kermany et al. (Kermany et al., 2018) |  5,840 |  Normal (1,575); Pneumonia Bacterial (2,771); non-COVID-19 lung infection (1,494)
K |  RSNA, Radiopedia and SIRM (Dadario, 2020) |  73 |  COVID-19
L |  RYDLS-20 (Pereira et al., 2020) |  1,144 |  Normal (1,000); Pneumonia (144): (MERS (10); COVID-19 (90); Pneumocystis (11); SARS (11); Streptococcus (12); Varicella (10))
M |  COVID-19 Radiography Database (Qatar university) (Rahman et al., 2020) |  21,165 |  COVID-19 (3,616); Normal (10,192); Viral Pneumonia (1,345); Non-COVID-19 lung infection (6,012)
N |  NIH Chest X-ray (Wang et al., 2017) |  108,948 |  Atelectasis; Mass; Cardiomegaly; Nodule; Effusion; Normal; Infiltration; Pneumonia
O |  Chest X-ray Images (Pneumonia) Mooney (2018) |  5,863 |  Pneumonia (bacterial and viral); Normal
P |  COVID-19 dataset (Societa Italiana di Radiologia Medica e Interventistica, 2020) |  115 |  COVID-19
Q |  CHUAC dataset (De Moura et al., 2020) |  1,616 |  Normal (728); Pathological (648); COVID-19 (240)
R |  COVID-19 X rays (Dadario, 2020) |  79 |  COVID-19

###  3.2 Approaches for the automatic detection of COVID-19 using X-ray images

Table LABEL:table:art_details presents relevant information regarding the
methodology applied in each selected paper, the best results achieved, the AI
models used, the image databases involved, the classes, and the preprocessing
operations. The state-of-the-art networks most used are the VGG and ResNet
family and the preprocessing steps more prevalent are resizing, normalization
and cropping. Most of the models reported have high performances, however, the
experimental setups are not always clear due to most of the authors combine
multiple databases and then split the resulting set in the training,
validation, and test partitions without doing any further clarification. In
many cases, the testing of the model is done using very few images and none of
the analyzed works perform clinical validation of the methods.

Table 4: Implementation details and results of the selected papers for the detection of COVID-19 using chest X-ray images Article |  Models |  Database |  Classes |  Preprocessing |  Best Results
---|---|---|---|---|---
(Apostolopoulos and Mpesiana, 2020) |  VGG16; MobileNetV2; Inception; Xception; Inception-ResNet-V2 |  I, J |  COVID-19; Pneumonia; Normal |  Resize, 200 x 266 |  MobileNetV2:
Accuracy, 94.7%;
Sensitivity, 98.7%;
Specificity, 96.5%
(Ucar and Korkmaz, 2020) |  COVID Diagnosis-Net (based on SqueezeNet) |  I, O |  COVID-19; Pneumonia; Normal |  Resize, 227 x 227; Shuffled; Normalization with the mean subtracting operation; Conversion to RGB with 8-bit depth |  Accuracy, 98.3%;
Specificity, 99.1%;
F1, 98.3%
(Ozturk et al., 2020) |  Dark CovidNet (based on Darknet-19 model) |  F, I |  (COVID-19; Normal); (COVID-19; Pneumonia; Normal) |  N/A |  Binary Accuracy, 98.1%
(Toğaçar et al., 2020) |  MobileNetV2; SqueezeNet; Social Mimic optimization method; SVM |  I, M |  COVID-19; Pneumonia; Normal |  Convertion to jpg format; Fuzzy Color technique |  Overall accuracy, 98.3%;
COVID-19 Sensitivity, 99.3%;
COVID-19 Specificity, 99.4%
(Pereira et al., 2020) |  SVM; MLP; DT; RF; Hierarchical Clus-HMC |  I, N, L |  Normal; COVID-19; MERS; SARS; Varicella; Streptococcus; Pneumocystis |  Manual crop |  Multiclass:
COVID-19 class F1 score, 83.3%
Hierarchical:
COVID-19 class F1 score, 88.8%
(Apostolopoulos et al., 2020) |  MobileNetV2 |  I, P |  COVID-19; Edema; Effusion; Emphys; Fibrosis; Pneumonia; Normal |  Resize, 200 x 200 |  Accuracy between the seven classes of 87.7%
(Waheed et al., 2020) |  ACGAN; VGG16 |  I, M, R |  COVID-19; Normal |  Resize, 112 x 112 x 3; Normalization |  Using actual data:
Sensitivity, 69.0%;
Specificity, 95.0%;
Accuracy, 85.0%
Including synthetic images:
Sensitivity, 95.0%;
Specificity, 90.0%;
Accuracy, 97.0%
(Khan et al., 2020) |  CoroNet |  I, O |  (COVID-19; Normal; Pneumonia bacterial; Pneumonia viral); (COVID-19; Pneumonia; Normal) |  Resize, 224 x 224 |  Four classes:
Accuracy, 93.0%;
Three classes:
Accuracy, 95.0%
(Das et al., 2020) |  Truncated Inception Net |  I, O, S |  COVID-19; Pneumonia; Tuberculosis; Normal |  Resize, 224 x 224 x 3 |  COVID-19 positive cases:
Accuracy, 99.96%;
AUC, 100%
(Toraman et al., 2020) |  CapsNet |  I, N |  (COVID-19; Normal); (COVID-19; Normal; Pneumonia) |  Resize, 128 x 128; Data augmentation |  Binary class:
Accuracy, 97.2%
Multi-class:
Accuracy, 84.2%
(Blain et al., 2021) |  U-Net; DenseNet121 |  K |  N/A |  Lung segmentation |  Diagnosing alveolar opacities:
Accuracy, 78.5%
Diagnosing interstitial opacities:
Accuracy, 90.7%
(Horry et al., 2020) |  VGG19 |  I, N |  COVID-19; Normal; Pneumonia |  Resize, 224 x 224; Histogram equalization using N-CLAHE |  X-ray:
Accuracy, 86.0%
Ultrasound:
Accuracy, 100%
CT:
Accuracy, 84.0%
(King et al., 2020) |  Self-Organizing Feature Map |  I |  COVID-19; Normal |  Resize |  Euclidean distance of 1.1 between 1st and 2nd winning neurons
(Karar et al., 2020) |  VGG16; ResNet50V2; DenseNet169 |  I |  Normal; COVID-19; Viral Pneumonia; Bacterial Pneumonia |  Resize, 150 x 150 |  Accuracy, 99.9%
(Ohata et al., 2021) |  MobileNet; DenseNet121; Inception-ResNet-V2; Bayes; RF; MLP; KNN; SVM |  I, R, O, N |  COVID-19; Normal |  Resize, (224 x 224; 299 x 299; 331 x 331) |  MobileNet + SVM (Linear):
Accuracy, 98.6%;
F1-score, 98.5%
(Shorfuzzaman and Hossain, 2020) |  Siamese Network |  I, O |  COVID-19; Normal; Pneumonia |  Resize; Normalization; Histogram-equalization |  Accuracy, 95.6%;
AUC, 98.9%
(De Moura et al., 2020) |  DenseNet161 |  Q |  COVID-19; Pathological; Normal; Combinations |  N/A |  Accuracy, 90.3% in (Normal & Pathological) vs. COVID-19
(Nayak et al., 2021b) |  AlexNet; VGG16; GoogleNet; MobileNetV2; SqueezeNet; ResNet34; ResNet50; InceptionV3 |  F, I |  COVID-19; Normal |  Normalization |  ResNet34, 98.3%
(Karakanis and Leontidis, 2021) |  CNN Designed |  I, O |  (COVID-19; Normal); (COVID-19; Normal; Bacterial Pneumonia) |  Resize, 224 x 224 |  Binary classification:
Accuracy, 98.7%;
Sensitivity, 100%;
Specificity, 98.3%
(Albahli and Yar, 2021) |  NasNetLarge; Xception; InceptionV3; Inception-ResNetV2; ResNet50 |  I, N |  COVID-19; normal; 14 other chest diseases |  Histogram equalization; Lung and heart segmentation |  First classifier:
Accuracy, 96.3%
Second classifier:
Accuracy, 87.8%
(Singh et al., 2021) |  VGG19; VGG16; ResNet50; DenseNet161; DenseNet169; Naive Bayes |  Q, R, M, N, G |  COVID-19; Pneumonia; Normal |  Histogram equalization (CLAHE); Dynamic image filtering (NLMD) |  Accuracy, 98.7%
(Sheykhivand et al., 2021) |  GANs; LSTM networks |  O, R, I, M |  (Normal; COVID-19); (Normal; Pneumonia (Viral; Bacterial; COVID19)); (Normal; COVID-19; Pneumonia Viral and Pneumonia Bacterial); (Normal; COVID-19; Pneumonia Bacterial); (Normal; COVID-19; Pneumonia Viral); (COVID-19; Pneumonia Bacterial; Pneumonia Viral); |  Resize, 224 x 224; Normalization |  Two classes: Accuracy, 99.5%
(Tuncer et al., 2021) |  SVM |  I, O |  COVID-19; Pneumonia; Normal |  Fuzzy transform |  Accuracy, 97.0%

We summarize the different approaches in the literature for COVID-19 automatic
detection using X-ray images. We aim to present the different strategies for
preprocessing, classification, and interpretability implemented in literature.

####  3.2.1 Preprocessing strategies

The most common preprocessing methods found are: normalization, cropping and
resizing, being the predominant size 224 x 224 (Sarv Ahrabi et al., 2021). It
has also been applied the selection of only anteroposterior (AP) or
posteroanterior (PA) views from the images in the databases (Arias-Londono et
al., 2020; Panwar et al., 2020) and the state-of-the-art architectures own
preprocessing (Castiglioni et al., 2021). Other strategies found include the
Fuzzy Color technique (Toğaçar et al., 2020) and the multiscale offline
augmentation, which incorporate shearing the image, adding Gaussian noise, and
decreasing the brightness (Ucar and Korkmaz, 2020).

Authors have stated that the CNNs used for the detection of COVID-19 using
chest X-ray images, base their classifications in areas in the input image
outside the region of interest, which has no relation with COVID-19 signs
(Majeed et al., 2020). Therefore, some works perform a segmentation of the
lungs using the U-Net network as a preprocessing step to ensure that the
network classification is based on regions inside them (Tartaglione et al.,
2020; Arias-Londono et al., 2020; Rajaraman et al., 2020); another approach is
presented by (Aslan et al., 2021) here the lungs are segmented before the
classification using a new ANN-based network.

Several authors include data augmentation in the network training to mitigate
the dataset imbalance, the most common transformations reported are rotation,
horizontal, and vertical flip (Khan et al., 2020; Zebin and Rezvy, 2021).
Other transformations applied are: Gaussian noise (Nayak et al., 2021b);
shearing, elastic distortion (Sarv Ahrabi et al., 2021) and histogram
equalization (Tartaglione et al., 2020). Data imbalance has also been
compensated using Generative Adversary Networks (GANs) to generate artificial
images and augment the minority COVID-19 class, some examples are the CycleGAN
(Karakanis and Leontidis, 2021); the conditional generative adversarial
networks (Zebin and Rezvy, 2021), and the Auxiliary Classifier Generative
Adversarial Network (ACGAN) (Waheed et al., 2020), this work shows that their
model accuracy can increase by 10% if the synthetic images produced are used
during training.

####  3.2.2 Classification methods

Despite the short time since COVID-19 emerged, the limited availability of
data and knowledge in this regard, many methods have been proposed for the
automatic detection of this disease using chest X-ray images.

Transfer learning using state-of-the-art CNNs architectures is the most
commonly found strategy to perform this task. Among the most frequently used
networks are VGG19, VGG16, Inception-ResNetV2, MobileNetV2, ResNet50, and
EfficientNetB0 (Apostolopoulos and Mpesiana, 2020; Zebin and Rezvy, 2021).

Figure 4 shows the VGG19 architecture that has a total of 19 layers, namely 16
2D-Convolutional layers and 3 dense layers. This figure shows the number of
filters, the kernel size, the strides size, the padding name, the activation
function, and the shape of the output of each layer. Usually, to implement
transfer learning, researchers deleted the layers after the Flatten, i.e., the
dense layers and Softmax activation function, and replace them with a
particular fully connected layer. The final activation function generates
predictions from the last dense layer, with a specific number of classes.

Figure 4: Original architecture of the VGG19 convolutional neural network.

Other state-of-the-art CNNs architectures have also been used in literature:
DenseNet161 is adapted to classify chest X-ray images acquired by portable
equipment (De Moura et al., 2020); ResNet50 and ResNet101 are used in a two-
stage model, where pneumonia and healthy images are initially identified and
later COVID-19 examples are classified from the pneumonia cases (Jain et al.,
2020); ResNet50 lead to the best results of a two stages model to classify
COVID-19 and other 14 chest diseases, being the results competitive with
currently used state-of-the-art models (Albahli and Yar, 2021); Inception is
truncated at a layer that is chosen experimentally to avoid possible
overfitting due to the lack of COVID-19 positive samples (Das et al., 2020).
Similarly, ImageNet pre-trained models are also pruned and their predictions
are combined through different ensemble strategies (Rajaraman et al., 2020).

A common approach uses state-of-the-art CNNs as feature extractors and machine
learning classifiers to make the final predictions, this integrated approach
has been performed using machine learning models like Support Vector Machine
(SVM), linear kernel and radial basis function, k-nearest neighbor, Decision
Tree, CN 2 rule inducer techniques and deep learning models like MobileNetV2,
ResNet50, GoogleNet, DarkNet, and Xception (Ohata et al., 2021; Mohammed et
al., 2021). Additional models are also used, namely; MobileNetV2 and
SqueezeNet are used to extract characteristics that are later processed using
the Social Mimic optimization method and the final classification task is
performed by a SVM (Toğaçar et al., 2020). A similar approach is made by
Ismael and Şengür (2021) using VGG16, VGG19, ResNet18, ResNet50, and ResNet101
and SVM. In (Sahlol et al., 2020) Inception is used to extract features, and
Marine Predators Algorithm, a swarm-based feature selection algorithm is used
to select the most relevant features; a fuzzy tree transformation is applied
to each chest image and then exemplar division, a novel machine learning model
is used. Then features are obtained using a multi-kernel local binary pattern,
the most important are selected using the iterative neighborhood component and
finally, conventional classifiers perform the classification. The best
performance is obtained using a cubic SVM (Tuncer et al., 2021).

Implementations of traditional machine learning methods and strategies like
late fusion, early fusion, and hierarchical classification are used to
classify not only COVID-19 but also up to 14 other lung diseases (Pereira et
al., 2020; Yoo et al., 2020).

Other methodologies were also found, some of the most representative are: an
ensemble of ten convolutional neural networks based on ResNet50 architecture
and ML models (Singh et al., 2021; Castiglioni et al., 2021); a cascaded
classification scheme using pre-trained CNN architectures (Karar et al.,
2020); a multi-kernel CNN block combined with pre-trained ResNet34 to overcome
imbalance in the dataset (Mursalim and Kurniawan, 2021); an integration of
contrastive learning with a fine-tuned pre-trained ConvNet encoder to capture
unbiased feature representations and a Siamese network, which makes the final
classification (Shorfuzzaman and Hossain, 2020); an unsupervised network
called Self-Organizing Feature Maps, which is analyzed using the saliency of
features, the authors state that the unsupervised method can extract features
that allow to accurately classify the COVID-19 chest X-ray images (King et
al., 2020); a multimodal approach using clinical and radiographic features,
both are compared using the unpaired student’s t-test or Mann-Whitney U test
and the segmentation and detection of opacities are also carried out (Blain et
al., 2021).

Many authors also implement their own networks, some of them are based on
state-of-the-art CNNs, such as VGG16 (Panwar et al., 2020), AlexNet (Aslan et
al., 2021) and Xception (Narayan Das et al., 2020). In (Ozturk et al., 2020) a
network based on DarkNet is evaluated by a radiologist, who concludes that the
model has a good performance detecting COVID-19 cases for the binary class
task, but it makes incorrect predictions in poor quality chest X-ray images
and patients with acute respiratory distress syndrome. The CoroNet network is
proposed based on the Xception architecture pre-trained on the ImageNet
dataset. It is trained using X-ray images of COVID-19 and other pneumonia, the
obtained network weights are publically available (Khan et al., 2020).
COVIDiagnosis-Net includes fine-tuning of the SqueezeNet using a bayesian
optimization method and offline augmentation only to COVID-19 class (Ucar and
Korkmaz, 2020).

Some authors propose their architectures from scratch, Sarv Ahrabi et al.
(2021) propose a network with 12 layers including convolution, max-pooling,
batch normalization, dropout, activation, and fully-connected layers; Hussain
et al. (2021) present a 22-layer CNN model, that is evaluated by a clinician
in multiple classification scenarios, 2 classes, 3 classes, and 4 classes.

Most of the found studies conduct a binary classification, COVID-19 vs Normal
cases. However, other works attempt multiclass classification, some of them
include: COVID-19, pneumonia and no-findings (Ucar and Korkmaz, 2020; Toraman
et al., 2020); COVID-19, pneumonia viral, pneumonia bacterial and no-findings
(Khan et al., 2020), (Jain et al., 2020); COVID-19, tuberculosis and non-
findings (Das et al., 2020; Yoo et al., 2020), and COVID-19 severity
classification (Blain et al., 2021). Some authors explore multiple
combinations of those scenarios (Hussain et al., 2021; Sheykhivand et al.,
2021; Majeed et al., 2020), and others take advantage of the availability of
other lung-related decease labels in the datasets and perform the
classification of COVID-19 examples along with multiple other pulmonary
diseases (Apostolopoulos et al., 2020; Albahli and Yar, 2021).

####  3.2.3 Interpretability and CNN benchmarking

Nayak et al. (2021b), analyze the performance of eight state-of-the-art CNNs,
they tune the number of trainable layers of the network, nodes, epochs, layers
of the classifier placed at the top of the network, the batch size, the
learning rate, and the optimizer algorithm, they find that the ResNet family
of architectures has the highest classification accuracy; a similar analysis
is implemented by (Majeed et al., 2020) using transfer learning and 12 state-
of-the-art CNNs architectures, a critical analysis that includes the needed
time to train each network is also presented, in the binary scenario the best
results were obtained using the networks Xception, Inception-ResnetV2, and
SqueezeNet; likewise, Apostolopoulos and Mpesiana (2020) implement transfer
learning to compare 5 state-of-the-art CNNs, the best result are obtained
using the VGG19.

Pham (2021), compare the performance of state-of-the-art fine-tuned CNNs and
recently developed networks like CovidGAN, CoroNet, and DarkCovidNet, using an
experimental setup as similar as possible to the original studies. Despite
exact comparisons are not possible due to databases updates, it is concluded
that similar performances are obtained using relatively small CNNs like
AlexNet or SqueeNet and the new and sometimes more complex architectures.
Tartaglione et al. (2020), provide insights and also raise warnings regarding
the generality of the results of COVID-19 classification using deep learning
and chest X-ray images. Alternately Horry et al. (2020) compare the results
achieved training the VGG16 network using CT, X-ray, and Ultrasound chest
images to classify COVID-19, pneumonia and healthy subjects. The best result
is achieved using Ultrasound images.

One of the main drawbacks of deep learning is the lack of interpretability,
which has imitated its application in some areas (Singh et al., 2020). That is
why, some authors have included visualization methods that could provide
credibility and increase trust in users. Particular focus has been given to
heat maps. In (Ozturk et al., 2020) is proposed a network that provides a heat
map along with the classifications, results are evaluated by a radiologist who
concludes that ‘The heat map showed a greater concentration area in the X-rays
of patients with COVID-19 than the area in which the disease is not seen’.
Besides heat maps, t-SNE is also used to improve explainability (Arias-Londono
et al., 2020), as well as class activation maps (Ucar and Korkmaz, 2020).

Finally, a widely used and accepted visualization method is the grand cam,
which is used in multiple works (Singh et al., 2021; Liang et al., 2021).

###  3.3 Metrics used in the evaluation of algorithms

Most of the metrics defined below are expressed for binary classification
tasks in terms of the numbers of True Positive (TP) predictions, True
Negatives (TN), False Positives (FP), and False Negatives (FN). TP refers to
the positive instances correctly classified as positive; TN refers to the
negative instances correctly classified as negative; FP refers to the negative
instances incorrectly classified as positive; FN refers to the positive
instances incorrectly classified as negative.

  * •

Accuracy: proportion of correct predictions, usually presented as a percentage
or as a number from 0 to 1.

$Accuracy=\frac{TP+TN}{TP+TN+FP+FN}$ (1)

  * •

Specificity: it measures the ability of the classifier to correctly identify
the negative instances of a class.

$Specificity=\frac{TN}{TN+FP}$ (2)

  * •

Recall/Sensitivity: also known as recall, it measures a classifier’s ability
to correctly identify the positive instances of a class.

$Recall=\frac{TP}{TP+FN}$ (3)

  * •

Precision/Positive Predictive Value: also known as precision, it represents
the proportion of positive cases among the instances classified as positive.

$Precision=\frac{TP}{TP+FP}$ (4)

  * •

F1-score: weighted harmonic mean of specificity and sensitivity normalized
between 0 and 1. This metric considers imbalanced classes, and it is useful
when a task requires high specificity and sensitivity.

$F1=2\frac{Precision~{}Recall}{Precision+Recall}$ (5)

  * •

Kappa statistics (K): standard measure of agreement between the expected and
the observed number of correct predictions. The expected number of correct
predictions is computed from class probabilities, making it suitable for
evaluating multiple classes in imbalanced datasets. The equation represents
the expected number of agreements as Pe and the observed as Po.

$K=\frac{Po-Pe}{1-Pe}$ (6)

##  4 Conclusions and future work

Given the worldwide impact of the COVID-19 pandemic, it is a priority to
develop alternative diagnostic tools available for everyone and offer reliable
results. This paper presents a systematic literature review of AI applied to
COVID-19 diagnosis using chest X-ray imaging. It includes the different
preprocessing techniques, classification methods using ML algorithms,
strategies to increase the interpretability of the models, and the articles
that perform a critical analysis of the state-of-the-art and the new
architecture designed to perform this task.

The main limitation researchers have faced when developing these systems is
the quality and availability of data. To overcome this situation, the use of
preprocessing techniques, such as histogram equalization, lung segmentation,
data augmentation using rotation or cropping operations, and synthetic data
generation using GANs, have been implemented to improve the detection
performances of the models. The most frequent approach for the classification
of COVID-19 from X-ray images is transfer learning using pre-trained CNNs
architectures, such as VGG16, DenseNet121, and ResNet50. Other proposals
involve state-of-the-art architectures as feature extractors and traditional
ML methods as classifiers. There are also multiple novel CNNs explicitly
designed for this task which allows more flexibility and potentially smaller
networks by narrowing the scope of the classification task. Overall,
literature methods have exceptional results with classification accuracies
over 95% and even 98%, however, the test set and the quality of the data, are
usually unclear.

Regarding the questions set at the start of the paper, we have shown that (1)
conventional image classification CNNs with pre-trained weights using
ImageNet, or more complex approaches as Capsule or Siamese networks have been
used to diagnose COVID-19 from chest X-ray images; (2) current detection
percentages are over 98% accuracy in binary classification (COVID-19 and
Normal). However, no clinical trials have been performed in none of these
models and the experimental setups are usually unclear; (3) this paper
compiles an active set of databases for training and evaluating AI models,
despite the relatively high number of available databases of chest X-rays,
there is a limited amount of labeled COVID-19 cases, which leads researchers
to combine various databases; and in (4) most of the papers that use lung
segmentation as a preprocessing step, do so using a U-Net architecture.

Based on the present literature review, we identify possible research
opportunities as follows:

  * •

Construct or contribute to databases of chest X-ray images aiming to create a
representation of the different characteristics of real-world images, allowing
proper benchmarking and future model proposals.

  * •

Develop new CNNs for image segmentation, focusing on the segmentation of lungs
and radiological findings associated with COVID-19 disease.

  * •

Broaden the classification scope to detect factors such as severity, or
disease progression.

  * •

Design new preprocessing operations or pipelines, taking into account, for
example, the removal of artifacts or medical devices such as necklaces, tubes,
or ECG lead wires.

  * •

Detect COVID-19, and its outcome considering other clinical variables such as
the patient’s history.

  * •

Perform transfer learning based on networks that have been trained for other
lung diseases.

  * •

Design architectures or computational elements of CNNs for the detection of
COVID-19 from validated reference models.

## Acknowledments

The authors acknowledge to SES Hospital Universitario de Caldas, Alcaldía de
Manizales, the Universidad Autónoma de Manizales (UAM) and Minciencias from
Colombia, and the Mixed Unit of Biomedical Imaging FISABIO-CIPF from Spain,
for their contributions and funding through the project “Detección de COVID-19
en imágenes de rayos X usando redes neuronales convolucionales” with code
699-106 from UAM and contract 831 from grant 874-2020 of Minciencias.

## Disclosure Statement

No potential conflict of interest was reported by the authors.

## Appendix

Table 5 shows some acronyms and their meanings.

Table 5: Acronyms Acronym | Description
---|---
ACGAN | Auxiliary Classifier Generative Adversarial Network
AI | Artificial Intelligence
CNNs | Convolutional Neural Networks
CT | Computer tomography
DL | Deep Learning
FN | False Negatives
FP | False Positives
GANs | Generative Adversarial Network
K | Kappa statistics
ML | Machine Learning
RT-PCR | Real-time reverse transcription polymerase chain reaction
SARS-CoV-2 | Severe Acute Respiratory Syndrome coronavirus 2
SVM | Support Vector Machine
TN | True Negatives
TP | True Positive
WHO | World Health Organization

## References

  * Wang et al. [2020] Chen Wang, Peter W Horby, Frederick G Hayden, and George F Gao.  A novel coronavirus outbreak of global health concern.  _The Lancet_ , 395(10223):470–473, 2 2020.  ISSN 1474-547X.  doi:10.1016/S0140-6736(20)30185-9.  URL http://www.ncbi.nlm.nih.gov/pubmed/31986257http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC7135038.
  * Yuen et al. [2020] Kit San Yuen, Zi Wei Ye, Sin Yee Fung, Chi Ping Chan, and Dong Yan Jin.  SARS-CoV-2 and COVID-19: The most important research questions.  _Cell and Bioscience_ , 10(40), 2020.  ISSN 20453701.  doi:10.1186/s13578-020-00404-4.  URL https://cellandbioscience.biomedcentral.com/articles/10.1186/s13578-020-00404-4.
  * World Health Organization [2020a] World Health Organization.  Coronavirus.  https://www.who.int/en/health-topics/coronavirus#tab=tab_1, 2020a.  Accesed on March 2021.
  * World Health Organization [2020b] World Health Organization.  WHO Director-General’s opening remarks at the media briefing on COVID-19.  https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19—11-march-2020, 3 2020b.  Accesed on 11 March 2021.
  * World Health Organization [2021] World Health Organization.  Covid-19 weekly epidemiological update as of 21 march 2021.  https://www.who.int/docs/default-source/coronaviruse/situation-reports/weekly_epidemiological_update_22.pdf, 2021\.  Accesed on 15 Apri 2021.
  * Anis et al. [2020] S Anis, K W Lai, J H Chuah, S M Ali, H Mohafez, M Hadizadeh, D Yan, and Z C. Ong.  An Overview of Deep Learning Approaches in Chest Radiograph.  _IEEE Access_ , 8:182347–182354, 2020.  doi:10.1109/ACCESS.2020.3028390.
  * Ng et al. [2020] Ming-Yen Ng, Elaine Y. P. Lee, Jin Yang, Fangfang Yang, Xia Li, Hongxia Wang, Macy Mei-sze Lui, Christine Shing-Yen Lo, Barry Leung, Pek-Lan Khong, Christopher Kim-Ming Hui, Kwok-yung Yuen, and Michael D. Kuo.  Imaging Profile of the COVID-19 Infection: Radiologic Findings and Literature Review.  _Radiology: Cardiothoracic Imaging_ , 2(1):e200034, 2020.  doi:10.1148/ryct.2020200034.
  * Yüce et al. [2021] Meral Yüce, Elif Filiztekin, and Korin Gasia Özkaya.  COVID-19 diagnosis —A review of current methods.  _Biosensors and Bioelectronics_ , 172(112752), 2021.  ISSN 18734235.  doi:10.1016/j.bios.2020.112752.  URL /pmc/articles/PMC7584564//pmc/articles/PMC7584564/?report=abstracthttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC7584564/.
  * Afzal [2020] Adeel Afzal.  Molecular diagnostic technologies for COVID-19: Limitations and challenges.  _Journal of Advanced Research_ , 26:149–159, 2020.  ISSN 20901232.  doi:10.1016/j.jare.2020.08.002.  URL https://doi.org/10.1016/j.jare.2020.08.002.
  * Aziz et al. [2020] Asma B. Aziz, Rubhana Raqib, Wasif A. Khan, Mahbubur Rahman, Rashidul Haque, Munir Alam, K. Zaman, and Allen G. Ross.  Integrated control of COVID-19 in resource-poor countries.  _International Journal of Infectious Diseases_ , 101(2020):98–101, 2020.  ISSN 18783511.  doi:10.1016/j.ijid.2020.09.009.
  * Fan et al. [2020] D P. Fan, T Zhou, G P. Ji, Y Zhou, G Chen, H Fu, J Shen, and L Shao.  Inf-Net: Automatic COVID-19 Lung Infection Segmentation From CT Images.  _IEEE Transactions on Medical Imaging_ , 39(8):2626–2637, 2020.  doi:10.1109/TMI.2020.2996645.
  * Ohata et al. [2021] Elene Firmeza Ohata, Gabriel Maia Bezerra, Joao Victor Souza Das Chagas, Aloisio Vieira Lira Neto, Adriano Bessa Albuquerque, Victor Hugo C.De Albuquerque, and Pedro Pedrosa Reboucas Filho.  Automatic detection of COVID-19 infection using chest X-ray images through transfer learning.  _IEEE/CAA Journal of Automatica Sinica_ , 8(1):239–248, 2021.  ISSN 23299274.  doi:10.1109/JAS.2020.1003393.
  * Ai et al. [2020] Tao Ai, Zhenlu Yang, Hongyan Hou, Chenao Zhan, Chong Chen, Wenzhi Lv, Qian Tao, Ziyong Sun, and Liming Xia.  Correlation of Chest CT and RT-PCR Testing for Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases.  _Radiology_ , 296(2), 2020.
  * Wong et al. [2020] Ho Yuen Frank Wong, Hiu Yin Sonia Lam, Ambrose Ho Tung Fong, Siu Ting Leung, Thomas Wing Yan Chin, Christine Shing Yen Lo, Macy Mei Sze Lui, Jonan Chun Yin Lee, Keith Wan Hang Chiu, Tom Wai Hin Chung, Elaine Yuen Phin Lee, Eric Yuk Fai Wan, Ivan Fan Ngai Hung, Tina Poy Wing Lam, Michael D. Kuo, and Ming Yen Ng.  Frequency and Distribution of Chest Radiographic Findings in Patients Positive for COVID-19.  _Radiology_ , 296(2):E72–E78, 2020.  ISSN 15271315.  doi:10.1148/radiol.2020201160.
  * Breiding [2009] Matthew J. Breiding.  Radiation Dose Associated with Common Computed Tomography Examinations and the Associated Lifetime Attributable Risk of Cancer.  _Arch Intern Med_ , 63(169(22)):2078–2086, 2009\.  doi:10.1001/archinternmed.2009.427.Radiation.
  * Jaiswal et al. [2019] Amit Kumar Jaiswal, Prayag Tiwari, Sachin Kumar, Deepak Gupta, Ashish Khanna, and Joel J P C Rodrigues.  Identifying pneumonia in chest X-rays: A deep learning approach.  _Measurement_ , 145:511–518, 2019.  ISSN 0263-2241.  doi:https://doi.org/10.1016/j.measurement.2019.05.076.  URL https://www.sciencedirect.com/science/article/pii/S0263224119305202.
  * Narin et al. [2021] A. Narin, C. Kaya, and Z Pamuk.  Automatic detection of coronavirus disease (COVID-19) using X-ray images and deep convolutional neural networks.  _Pattern Anal Applic_ , 2021.  doi:https://doi.org/10.1007/s10044-021-00984-y.
  * Kaufman et al. [2020] Audrey E Kaufman, Sonum Naidu, Sarayu Ramachandran, Dalia S Kaufman, Zahi A Fayad, and Venkatesh Mani.  Review of radiographic findings in COVID-19.  _World Journal of Radiology_ , 12(8):142–155, 8 2020.  ISSN 1949-8470.  doi:10.4329/wjr.v12.i8.142.  URL /pmc/articles/PMC7457163//pmc/articles/PMC7457163/?report=abstracthttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC7457163/.
  * Yasin and Gouda [2020] Rabab Yasin and Walaa Gouda.  Chest X-ray findings monitoring COVID-19 disease course and severity.  _Egyptian Journal of Radiology and Nuclear Medicine_ , 51(1), 12 2020.  ISSN 2090-4762.  doi:10.1186/s43055-020-00296-x.
  * Cleverley et al. [2020] Joanne Cleverley, James Piper, and Melvyn M. Jones.  The role of chest radiography in confirming covid-19 pneumonia.  _The BMJ_ , 370, jul 2020.  ISSN 17561833.  doi:10.1136/bmj.m2426.  URL http://dx.doi.org/10.1136/bmj.m2426.
  * Rousan et al. [2020] Liqa A. Rousan, Eyhab Elobeid, Musaab Karrar, and Yousef Khader.  Chest x-ray findings and temporal lung changes in patients with COVID-19 pneumonia.  _BMC Pulmonary Medicine_ , 20(1:245), 9 2020.  ISSN 14712466.  doi:10.1186/s12890-020-01286-5.  URL https://bmcpulmmed.biomedcentral.com/articles/10.1186/s12890-020-01286-5.
  * WHO Solidarity Trial Consortium [2021] WHO Solidarity Trial Consortium.  Repurposed Antiviral Drugs for Covid-19 — Interim WHO Solidarity Trial Results, 2021.  ISSN 0028-4793.
  * WHO [2021] WHO.  7 Vaccines Approved for Use by WHO.  https://covid19.trackvaccines.org/agency/who/, 2021.  Accesed on 18 october 2021.
  * Arteaga-Arteaga et al. [2022] Harold Brayan Arteaga-Arteaga, Melissa DelaPava, Alejandro Mora-Rubio, Mario Alejandro Bravo-Ortíz, Jesus Alejandro Alzate-Grisales, Daniel Arias-Garzón, Luis Humberto López-Murillo, Felipe Buitrago-Carmona, Juan Pablo Villa-Pulgarín, Esteban Mercado-Ruiz, Fernanda Martínez Rodríguez, Maria Jose Palancares Sosa, Sonia H. Contreras-Ortiz, Simon Orozco-Arias, Mahmoud Hassaballah, María de la Iglesia Vayá, Oscar Cardona-Morales, and Reinel Tabares-Soto.  Deep Learning Applied to COVID-19 Detection in X-Ray Images.  In _AI Applications for Disease Diagnosis and Treatment_ , pages 202–247. IGI Global, 6 2022.  doi:10.4018/978-1-6684-2304-2.ch007.  URL https://services.igi-global.com/resolvedoi/resolve.aspx?doi=10.4018/978-1-6684-2304-2.ch007.
  * Goodfellow et al. [2016] Ian Goodfellow, Yoshua Bengio, and Aaron Courville.  _Deep Learning_.  MIT Press, 2016.
  * Orozco-Arias et al. [2019] Simon Orozco-Arias, Gustavo Isaza, Romain Guyot, and Reinel Tabares-Soto.  A systematic review of the application of machine learning in the detection and classification of transposable elements.  _PeerJ_ , 7:e8311, 12 2019.  doi:10.7717/peerj.8311.
  * Tabares-Soto et al. [2019] Reinel Tabares-Soto, Ramos Pollán Raúl, and Isaza Gustavo.  Deep learning applied to steganalysis of digital images: A systematic review.  _IEEE Access_ , 7:68970–68990, 2019.  ISSN 21693536.  doi:10.1109/ACCESS.2019.2918086.
  * Bravo Ortíz et al. [2021] Mario Alejandro Bravo Ortíz, Harold Brayan Arteaga Arteaga, Reinel Tabares Soto, Jorge Iván Padilla Buriticá, and Simon Orozco Arias.  Cervical cancer classification using convolutional neural networks, transfer learning and data augmentation.  _Revista EIA_ , 18(35), 2021.  ISSN 1794-1237.  doi:10.24050/reia.v18i35.1462.  URL https://revista.eia.edu.co/index.php/reveia/article/view/1462.
  * Reinel et al. [2021] T. S. Reinel, A. A. H. Brayan, B. O. M. Alejandro, M. R. Alejandro, A. G. Daniel, A. G. J. Alejandro, B. J. A. Buenaventura, O. A. Simon, I. Gustavo, and R. P. Raúl.  GBRAS-Net: A Convolutional Neural Network Architecture for Spatial Image Steganalysis.  _IEEE Access_ , 9:14340–14350, 2021.  doi:10.1109/ACCESS.2021.3052494.
  * Arteaga-Arteaga et al. [2021] Harold Brayan Arteaga-Arteaga, Alejandro Mora-Rubio, Frank Florez, Nicolas Murcia-Orjuela, Cristhian Eduardo Diaz-Ortega, Simon Orozco-Arias, Melissa delaPava, Mario Alejandro Bravo-Ortíz, Melvin Robinson, Pablo Guillen-Rondon, and Reinel Tabares-Soto.  Machine learning applications to predict two-phase flow patterns.  _PeerJ Computer Science_ , 7:e798, 2021.  ISSN 23765992.  doi:10.7717/peerj-cs.798.
  * Amisha et al. [2019] Amisha, Paras Malik, Monika Pathania, and VyasKumar Rathaur.  Overview of artificial intelligence in medicine.  _Journal of Family Medicine and Primary Care_ , 8(7):2328–2331, 2019.  ISSN 2249-4863.  doi:10.4103/jfmpc.jfmpc_440_19.  URL /pmc/articles/PMC6691444//pmc/articles/PMC6691444/?report=abstracthttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC6691444/.
  * Ahuja [2019] Abhimanyu S. Ahuja.  The impact of artificial intelligence in medicine on the future role of the physician.  _PeerJ_ , 7(e7702), 2019.  ISSN 21678359.  doi:10.7717/peerj.7702.
  * Pham [2021] Tuan D. Pham.  Classification of COVID-19 chest X-rays with deep learning: new models or fine tuning?  _Health Information Science and Systems_ , 9(1), 12 2021\.  ISSN 2047-2501.  doi:10.1007/s13755-020-00135-3.
  * Hussain et al. [2021] Emtiaz Hussain, Mahmudul Hasan, Md Anisur Rahman, Ickjai Lee, Tasmi Tamanna, and Mohammad Zavid Parvez.  CoroDet: A deep learning based classification for COVID-19 detection using chest X-ray images.  _Chaos, Solitons and Fractals_ , 142(110495), 2021.  ISSN 09600779.  doi:10.1016/j.chaos.2020.110495.  URL https://doi.org/10.1016/j.chaos.2020.110495.
  * Hussain et al. [2020] Lal Hussain, Tony Nguyen, Haifang Li, Adeel A. Abbasi, Kashif J. Lone, Zirun Zhao, Mahnoor Zaib, Anne Chen, and Tim Q. Duong.  Machine-learning classification of texture features of portable chest X-ray accurately classifies COVID-19 lung infection.  _BioMedical Engineering Online_ , 19(1), 12 2020.  ISSN 1475925X.  doi:10.1186/s12938-020-00831-x.
  * Pereira et al. [2020] Rodolfo M. Pereira, Diego Bertolini, Lucas O. Teixeira, Carlos N. Silla, and Yandre M.G. Costa.  COVID-19 identification in chest X-ray images on flat and hierarchical classification scenarios.  _Computer Methods and Programs in Biomedicine_ , 194, 2020.  ISSN 18727565.  doi:10.1016/j.cmpb.2020.105532.
  * Else [2020] Holly Else.  How a torrent of COVID science changed research publishing - in seven charts.  _Nature_ , 588(553), dec 2020.  ISSN 14764687.  doi:10.1038/d41586-020-03564-y.
  * Rahman et al. [2021] Sejuti Rahman, Sujan Sarker, Md Abdullah Al Miraj, Ragib Amin Nihal, A. K. M. Nadimul Haque, and Abdullah Al Noman.  Deep Learning–Driven Automated Detection of COVID-19 from Radiography Images: a Comparative Analysis.  _Cognitive Computation_ , 2021.  ISSN 1866-9956.  doi:10.1007/s12559-020-09779-5.
  * Nayak et al. [2021a] Janmenjoy Nayak, Bighnaraj Naik, Paidi Dinesh, Kanithi Vakula, Pandit Byomakesha Dash, and Danilo Pelusi.  Significance of deep learning for Covid-19: state-of-the-art review.  _Research on Biomedical Engineering_ , 2021a.  ISSN 24464740.  doi:10.1007/s42600-021-00135-6.
  * Wynants et al. [2020] Laure Wynants, Ben Van Calster, Gary S Collins, Richard D Riley, Georg Heinze, Ewoud Schuit, Marc M J Bonten, Darren L Dahly, Johanna A Damen, Thomas P A Debray, Valentijn M T de Jong, Maarten De Vos, Paula Dhiman, Maria C Haller, Michael O Harhay, Liesbet Henckaerts, Pauline Heus, Michael Kammer, Nina Kreuzberger, Anna Lohmann, Kim Luijken, Jie Ma, Glen P Martin, David J McLernon, Constanza L Andaur Navarro, Johannes B Reitsma, Jamie C Sergeant, Chunhu Shi, Nicole Skoetz, Luc J M Smits, Kym I E Snell, Matthew Sperrin, René Spijker, Ewout W Steyerberg, Toshihiko Takada, Ioanna Tzoulaki, Sander M J van Kuijk, Bas C T van Bussel, Iwan C C van der Horst, Florien S van Royen, Jan Y Verbakel, Christine Wallisch, Jack Wilkinson, Robert Wolff, Lotty Hooft, Karel G M Moons, and Maarten van Smeden.  Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal.  _BMJ_ , 369:m1328, 2020.  doi:10.1136/bmj.m1328.  URL https://www.bmj.com/content/369/bmj.m1328.
  * Shi et al. [2021] Feng Shi, Jun Wang, Jun Shi, Ziyan Wu, Qian Wang, Zhenyu Tang, Kelei He, Yinghuan Shi, and Dinggang Shen.  Review of Artificial Intelligence Techniques in Imaging Data Acquisition, Segmentation, and Diagnosis for COVID-19.  _IEEE Reviews in Biomedical Engineering_ , 14:4–15, 2021\.  ISSN 19411189.  doi:10.1109/RBME.2020.2987975.
  * Swapnarekha et al. [2020] H. Swapnarekha, Himansu Sekhar Behera, Janmenjoy Nayak, and Bighnaraj Naik.  Role of intelligent computing in COVID-19 prognosis: A state-of-the-art review.  _Chaos, Solitons and Fractals_ , 138:109947, 2020.  ISSN 09600779.  doi:10.1016/j.chaos.2020.109947.  URL https://doi.org/10.1016/j.chaos.2020.109947.
  * Waleed Salehi et al. [2020] Ahmad Waleed Salehi, Preety Baglat, and Gaurav Gupta.  Review on machine and deep learning models for the detection and prediction of coronavirus.  _Materials Today: Proceedings_ , 33:3896–3901, 2020.  ISSN 2214-7853.  doi:https://doi.org/10.1016/j.matpr.2020.06.245.  URL https://www.sciencedirect.com/science/article/pii/S2214785320347131.  International Conference on Nanotechnology: Ideas, Innovation and Industries.
  * Albahri et al. [2020] O. S. Albahri, A. A. Zaidan, A. S. Albahri, B. B. Zaidan, Karrar Hameed Abdulkareem, Z. T. Al-qaysi, A. H. Alamoodi, A. M. Aleesa, M. A. Chyad, R. M. Alesa, L. C. Kem, Muhammad Modi Lakulu, A. B. Ibrahim, and Nazre Abdul Rashid.  Systematic review of artificial intelligence techniques in the detection and classification of COVID-19 medical images in terms of evaluation and benchmarking: Taxonomy analysis, challenges, future solutions and methodological aspects.  _Journal of Infection and Public Health_ , 13(10):1381–1396, 2020.  ISSN 1876035X.  doi:10.1016/j.jiph.2020.06.028.  URL https://doi.org/10.1016/j.jiph.2020.06.028.
  * Bansal et al. [2020] Agam Bansal, Rana Prathap Padappayil, Chandan Garg, Anjali Singal, Mohak Gupta, and Allan Klein.  Utility of Artificial Intelligence Amidst the COVID 19 Pandemic: A Review.  _Journal of Medical Systems_ , 44(156), 2020.
  * Syeda et al. [2021] Hafsa Bareen Syeda, Mahanazuddin Syed, Kevin Wayne Sexton, Shorabuddin Syed, Salma Begun, Farhanuddin Syed, Fred Prior, and Feliciano Yu.  Role of Machine Learning Techniques to Tackle the COVID-19Crisis:Systematic Review.  _JMIR MEDICAL INFORMATICS_ , 9(1), 2021.
  * Roberts et al. [2021] Michael Roberts, Derek Driggs, Matthew Thorpe, Julian Gilbey, Michael Yeung, Stephan Ursprung, Angelica I. Aviles-Rivero, Christian Etmann, Cathal McCague, Lucian Beer, Jonathan R. Weir-McCall, Zhongzhao Teng, Effrossyni Gkrania-Klotsas, James H. F. Rudd, Evis Sala, and Carola-Bibiane Schönlieb.  Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans.  _Nature Machine Intelligence_ , 3(3):199–217, 2021.  ISSN 25225839.  doi:10.1038/s42256-021-00307-0.
  * Moher [2009] David Moher.  Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement.  _Annals of Internal Medicine_ , 151(4):264, 8 2009\.  ISSN 0003-4819.  doi:10.7326/0003-4819-151-4-200908180-00135.  URL http://annals.org/article.aspx?doi=10.7326/0003-4819-151-4-200908180-00135.
  * Paul Cohen et al. [2020] Joseph Paul Cohen, Paul Morrison, and Lan Dao.  COVID-19 Image Data Collection.  https://ui.adsabs.harvard.edu/abs/2020arXiv200311597C/abstract, 2020.  ISSN 23318422.  Accesed on 10 Apri 2021.
  * HM Hospitals [2020] HM Hospitals.  Dataset “Covid Data Save lives”.  https://www.hmhospitales.com/coronavirus/covid-data-save-lives/english-version, 2020\.  URL https://www.hmhospitales.com/coronavirus/covid-data-save-lives/english-version.  Accesed on 15 Apri 2021.
  * Vayá et al. [2021] Maria delaIglesia Vayá, Jose Manuel Saborit-Torres, Joaquim Angel Montell Serrano, Elena Oliver-Garciaa, Antonio Pertusa, Aurelia Bustosa, Miguel Cazorlaa, Joaquin Galant, Xavier Barber, Domingo Orozco-Beltrán, Francisco García-García, Marisa Caparrós, Germán González, and Jose María Salinas.  Bimcv covid-19+: a large annotated dataset of rx and ct images from covid-19 patients, 2021.  URL https://dx.doi.org/10.21227/w3aw-rv39.
  * Actualmed et al. [2020] Actualmed, José Antonio Heredia Álvaro, and Pau Agustí Ballester.  Figure 1 COVID-19 Chest X-ray Dataset Initiative.  https://github.com/agchung/Figure1-COVID-chestxray-dataset, 2020.  Accesed on 03 Apri 2021.
  * Jaeger et al. [2014] Stefan Jaeger, Sema Candemir, Sameer Antani, Yì-Xiáng J Wáng, Pu-Xuan Lu, and George Thoma.  Two public chest X-ray datasets for computer-aided screening of pulmonary diseases.  _Quantitative imaging in medicine and surgery_ , 4(6):475–477, 2014.  ISSN 2223-4292.  doi:10.3978/j.issn.2223-4292.2014.11.20.
  * Wang et al. [2017] Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M. Summers.  ChestX-ray8: Hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases.  In _Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017_ , volume 2017-January, pages 3462–3471. Institute of Electrical and Electronics Engineers Inc., 11 2017.  ISBN 9781538604571.  doi:10.1109/CVPR.2017.369.
  * Irvin et al. [2019] Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, Jayne Seekins, David A. Mong, Safwan S. Halabi, Jesse K. Sandberg, Ricky Jones, David B. Larson, Curtis P. Langlotz, Bhavik N. Patel, Matthew P. Lungren, and Andrew Y. Ng.  CheXpert: A large chest radiograph dataset with uncertainty labels and expert comparison.  _33rd AAAI Conference on Artificial Intelligence, AAAI 2019, 31st Innovative Applications of Artificial Intelligence Conference, IAAI 2019 and the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019_ , pages 590–597, 2019.  ISSN 2159-5399.  doi:10.1609/aaai.v33i01.3301590.
  * Johnson et al. [2019] Alistair E.W. Johnson, Tom J. Pollard, Seth J. Berkowitz, Nathaniel R. Greenbaum, Matthew P. Lungren, Chih ying Deng, Roger G. Mark, and Steven Horng.  MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports.  _Scientific Data_ , 6(317), 2019.  ISSN 20524463.  doi:10.1038/s41597-019-0322-0.  URL http://dx.doi.org/10.1038/s41597-019-0322-0.
  * Kermany et al. [2018] Daniel S. Kermany, Michael Goldbaum, Wenjia Cai, Carolina C.S. Valentim, Huiying Liang, Sally L. Baxter, Alex McKeown, Ge Yang, Xiaokang Wu, Fangbing Yan, Justin Dong, Made K. Prasadha, Jacqueline Pei, Magdalena Ting, Jie Zhu, Christina Li, Sierra Hewett, Jason Dong, Ian Ziyar, Alexander Shi, Runze Zhang, Lianghong Zheng, Rui Hou, William Shi, Xin Fu, Yaou Duan, Viet A.N. Huu, Cindy Wen, Edward D. Zhang, Charlotte L. Zhang, Oulan Li, Xiaobo Wang, Michael A. Singer, Xiaodong Sun, Jie Xu, Ali Tafreshi, M. Anthony Lewis, Huimin Xia, and Kang Zhang.  Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning.  _Cell_ , 172(5):1122–1131, 2018.  ISSN 10974172.  doi:10.1016/j.cell.2018.02.010.  URL https://doi.org/10.1016/j.cell.2018.02.010.
  * Dadario [2020] Andrew M. Dadario.  X rays and CT snapshots of COVID-19 patients.  https://www.kaggle.com/andrewmvd/convid19-x-rays, 2020.  Accesed on 12 Apri 2021.
  * Rahman et al. [2020] Tawsifur Rahman, Muhammad Chowdhury, and Amith Khandakar.  COVID-19 Radiography Database | Kaggle.  https://www.kaggle.com/tawsifurrahman/covid19-radiography-database, 2020\.  Accesed on 14 Apri 2021.
  * Mooney [2018] Paul Mooney.  Chest X-Ray Images (Pneumonia) - Kaggle.  https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia/activity, 2018.  Accesed on 15 Apri 2021.
  * Societa Italiana di Radiologia Medica e Interventistica [2020] Societa Italiana di Radiologia Medica e Interventistica.  BASE DE DATOS COVID-19 | SIRM.  https://www.sirm.org/category/senza-categoria/covid-19/, 2020.  Accesed on 15 Apri 2021.
  * De Moura et al. [2020] Joaquim De Moura, Lucia Ramos Garcia, Placido Francisco Lizancos Vidal, Milena Cruz, Laura Abelairas Lopez, Eva Castro Lopez, Jorge Novo, and Marcos Ortega.  Deep Convolutional Approaches for the Analysis of COVID-19 Using Chest X-Ray Images From Portable Devices.  _IEEE Access_ , 8:195594–195607, 2020.  ISSN 2169-3536.  doi:10.1109/access.2020.3033762.
  * Apostolopoulos and Mpesiana [2020] Ioannis D. Apostolopoulos and Tzani A. Mpesiana.  Covid-19: automatic detection from X-ray images utilizing transfer learning with convolutional neural networks.  _Physical and Engineering Sciences in Medicine_ , 43(2):635–640, 6 2020.  ISSN 26624737.  doi:10.1007/s13246-020-00865-4.
  * Ucar and Korkmaz [2020] Ferhat Ucar and Deniz Korkmaz.  COVIDiagnosis-Net: Deep Bayes-SqueezeNet based diagnosis of the coronavirus disease 2019 (COVID-19) from X-ray images.  _Medical Hypotheses_ , 140(109761), 2020.  ISSN 15322777.  doi:10.1016/j.mehy.2020.109761.  URL https://doi.org/10.1016/j.mehy.2020.109761.
  * Ozturk et al. [2020] Tulin Ozturk, Muhammed Talo, Eylul Azra Yildirim, Ulas Baran Baloglu, Ozal Yildirim, and U. Rajendra Acharya.  Automated detection of COVID-19 cases using deep neural networks with X-ray images.  _Computers in Biology and Medicine_ , 121(103792), 2020\.  ISSN 18790534.  doi:10.1016/j.compbiomed.2020.103792.  URL https://doi.org/10.1016/j.compbiomed.2020.103792.
  * Toğaçar et al. [2020] Mesut Toğaçar, Burhan Ergen, and Zafer Cömert.  COVID-19 detection using deep learning models to exploit Social Mimic Optimization and structured chest X-ray images using fuzzy color and stacking approaches.  _Computers in Biology and Medicine_ , 121(103805), 2020\.  ISSN 18790534.  doi:10.1016/j.compbiomed.2020.103805.
  * Apostolopoulos et al. [2020] Ioannis D. Apostolopoulos, Sokratis I. Aznaouridis, and Mpesiana A. Tzani.  Extracting Possibly Representative COVID-19 Biomarkers from X-ray Images with Deep Learning Approach and Image Data Related to Pulmonary Diseases.  _Journal of Medical and Biological Engineering_ , 40(3):462–469, 2020.  ISSN 21994757.  doi:10.1007/s40846-020-00529-4.  URL https://doi.org/10.1007/s40846-020-00529-4.
  * Waheed et al. [2020] A Waheed, M Goyal, D Gupta, A Khanna, F Al-Turjman, and P R Pinheiro.  CovidGAN: Data Augmentation Using Auxiliary Classifier GAN for Improved Covid-19 Detection.  _IEEE Access_ , 8:91916–91923, 2020.  doi:10.1109/ACCESS.2020.2994762.
  * Khan et al. [2020] Asif Iqbal Khan, Junaid Latief Shah, and Mohammad Mudasir Bhat.  CoroNet: A deep neural network for detection and diagnosis of COVID-19 from chest x-ray images.  _Computer Methods and Programs in Biomedicine_ , 196, 11 2020.  ISSN 18727565.  doi:10.1016/j.cmpb.2020.105581.
  * Das et al. [2020] Dipayan Das, K. C. Santosh, and Umapada Pal.  Truncated inception net: COVID-19 outbreak screening using chest X-rays.  _Physical and Engineering Sciences in Medicine_ , 43(3):915–925, 2020.  ISSN 26624737.  doi:10.1007/s13246-020-00888-x.  URL https://doi.org/10.1007/s13246-020-00888-x.
  * Toraman et al. [2020] Suat Toraman, Talha Burak Alakus, and Ibrahim Turkoglu.  Convolutional capsnet: A novel artificial neural network approach to detect COVID-19 disease from X-ray images using capsule networks.  _Chaos, Solitons and Fractals_ , 140, 2020.  ISSN 09600779.  doi:10.1016/j.chaos.2020.110122.
  * Blain et al. [2021] Maxime Blain, Michael T. Kassin, Nicole Varble, Xiaosong Wang, Ziyue Xu, Daguang Xu, Gianpaolo Carrafiello, Valentina Vespro, Elvira Stellato, Anna Maria Ierardi, Letizia Di Meglio, Robert D. Suh, Stephanie A. Walker, Sheng Xu, Thomas H. Sanford, Evrim B. Turkbey, Stephanie Harmon, Baris Turkbey, and Bradford J. Wood.  Determination of disease severity in COVID-19 patients using deep learning in chest x-ray images.  _Diagnostic and Interventional Radiology_ , 27(1):20–27, 2021.  ISSN 13053612.  doi:10.5152/dir.2020.20205.
  * Horry et al. [2020] M J Horry, S Chakraborty, M Paul, A Ulhaq, B Pradhan, M Saha, and N Shukla.  COVID-19 Detection Through Transfer Learning Using Multimodal Imaging Data.  _IEEE Access_ , 8:149808–149824, 2020.  doi:10.1109/ACCESS.2020.3016780.
  * King et al. [2020] Bayley King, Siddharth Barve, Andrew Ford, and Rashmi Jha.  Unsupervised Clustering of COVID-19 Chest X-Ray Images with a Self-Organizing Feature Map.  _Midwest Symposium on Circuits and Systems_ , pages 395–398, 2020\.  ISSN 15483746.  doi:10.1109/MWSCAS48704.2020.9184493.
  * Karar et al. [2020] Mohamed Esmail Karar, Ezz El-Din Hemdan, and Marwa A. Shouman.  Cascaded deep learning classifiers for computer-aided diagnosis of COVID-19 and pneumonia diseases in X-ray scans.  _Complex Intelligent Systems_ , 7(1):235–247, 2020.  ISSN 2199-4536.  doi:10.1007/s40747-020-00199-4.  URL https://doi.org/10.1007/s40747-020-00199-4.
  * Shorfuzzaman and Hossain [2020] Mohammad Shorfuzzaman and M. Shamim Hossain.  MetaCOVID: A Siamese neural network framework with contrastive loss for n-shot diagnosis of COVID-19 patients.  _Pattern Recognition_ , 113(107700), 2020.  ISSN 00313203.  doi:10.1016/j.patcog.2020.107700.  URL https://doi.org/10.1016/j.patcog.2020.107700.
  * Nayak et al. [2021b] Soumya Ranjan Nayak, Deepak Ranjan Nayak, Utkarsh Sinha, Vaibhav Arora, and Ram Bilas Pachori.  Application of deep learning techniques for detection of COVID-19 cases using chest X-ray images: A comprehensive study.  _Biomedical Signal Processing and Control_ , 64(PMID: 33230398; PMCID: PMC7674150.), 2 2021b.  ISSN 17468108.  doi:10.1016/j.bspc.2020.102365.
  * Karakanis and Leontidis [2021] Stefanos Karakanis and Georgios Leontidis.  Lightweight deep learning models for detecting COVID-19 from chest X-ray images.  _Computers in biology and medicine_ , 130(104181), 3 2021\.  ISSN 1879-0534.  doi:10.1016/j.compbiomed.2020.104181.  URL http://www.ncbi.nlm.nih.gov/pubmed/33360271http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC7831681.
  * Albahli and Yar [2021] Saleh Albahli and Ghulam Nabi Ahmad Hassan Yar.  Fast and Accurate Detection of COVID-19 Along With 14 Other Chest Pathologies Using a Multi-Level Classification: Algorithm Development and Validation Study.  _Journal of Medical Internet Research_ , 23(2):e23693, 2021.  ISSN 14388871.  doi:10.2196/23693.  URL https://pubmed.ncbi.nlm.nih.gov/33529154/.
  * Singh et al. [2021] Rajeev Kumar Singh, Rohan Pandey, and Rishie Nandhan Babu.  COVIDScreen: explainable deep learning framework for differential diagnosis of COVID-19 using chest X-rays.  _Neural Computing and Applications_ , 6, 2021.  ISSN 14333058.  doi:10.1007/s00521-020-05636-6.  URL https://doi.org/10.1007/s00521-020-05636-6.
  * Sheykhivand et al. [2021] Sobhan Sheykhivand, Zohreh Mousavi, Sina Mojtahedi, Tohid Yousefi Rezaii, Ali Farzamnia, Saeed Meshgini, and Ismail Saad.  Developing an efficient deep neural network for automatic detection of COVID-19 using chest X-ray images.  _Alexandria Engineering Journal_ , 60(3):2885–2903, 2021.  ISSN 11100168.  doi:10.1016/j.aej.2021.01.011.  URL https://doi.org/10.1016/j.aej.2021.01.011.
  * Tuncer et al. [2021] Turker Tuncer, Fatih Ozyurt, Sengul Dogan, and Abdulhamit Subasi.  A novel Covid-19 and pneumonia classification method based on F-transform.  _Chemometrics and Intelligent Laboratory Systems_ , 210(104256), 2021.  ISSN 18733239.  doi:10.1016/j.chemolab.2021.104256.  URL https://doi.org/10.1016/j.chemolab.2021.104256.
  * Sarv Ahrabi et al. [2021] Sima Sarv Ahrabi, Michele Scarpiniti, Enzo Baccarelli, and Alireza Momenzadeh.  An accuracy vs. Complexity comparison of deep learning architectures for the detection of covid-19 disease.  _Computation_ , 9(1), 2021.  ISSN 20793197.  doi:10.3390/computation9010003.
  * Arias-Londono et al. [2020] Julian D. Arias-Londono, Jorge A. Gomez-Garcia, Laureano Moro-Velazquez, and Juan I. Godino-Llorente.  Artificial Intelligence applied to chest X-Ray images for the automatic detection of COVID-19. A thoughtful evaluation approach.  _IEEE Access_ , 8:226811–226827, 2020.  ISSN 21693536.  doi:10.1109/ACCESS.2020.3044858.
  * Panwar et al. [2020] Harsh Panwar, P. K. Gupta, Mohammad Khubeb Siddiqui, Ruben Morales-Menendez, and Vaishnavi Singh.  Application of deep learning for fast detection of COVID-19 in X-Rays using nCOVnet.  _Chaos, Solitons and Fractals_ , 138, 9 2020.  ISSN 09600779.  doi:10.1016/j.chaos.2020.109944.
  * Castiglioni et al. [2021] Isabella Castiglioni, Davide Ippolito, Matteo Interlenghi, Caterina Beatrice Monti, Christian Salvatore, Simone Schiaffino, Annalisa Polidori, Davide Gandola, Cristina Messa, and Francesco Sardanelli.  Machine learning applied on chest x-ray can aid in the diagnosis of COVID-19: a first experience from Lombardy, Italy.  _European Radiology Experimental_ , 5(1), 2021.  ISSN 25099280.  doi:10.1186/s41747-020-00203-z.
  * Majeed et al. [2020] Taban Majeed, Rasber Rashid, Dashti Ali, and Aras Asaad.  Issues associated with deploying CNN transfer learning to detect COVID-19 from chest X-rays.  _Physical and Engineering Sciences in Medicine_ , 43(4):1289–1303, 2020.  ISSN 26624737.  doi:10.1007/s13246-020-00934-8.  URL https://doi.org/10.1007/s13246-020-00934-8.
  * Tartaglione et al. [2020] Enzo Tartaglione, Carlo Alberto Barbano, Claudio Berzovini, Marco Calandri, and Marco Grangetto.  Unveiling COVID-19 from chest x-ray with deep learning: A hurdles race with small data.  _International Journal of Environmental Research and Public Health_ , 17(18), 9 2020.  ISSN 16604601.  doi:10.3390/ijerph17186933.
  * Rajaraman et al. [2020] Sivaramakrishnan Rajaraman, Jenifer Siegelman, Philip O. Alderson, Lucas S. Folio, Les R. Folio, and Sameer K. Antani.  Iteratively Pruned Deep Learning Ensembles for COVID-19 Detection in Chest X-Rays.  _IEEE Access_ , 8:115041–115050, 2020.  ISSN 21693536.  doi:10.1109/ACCESS.2020.3003810.
  * Aslan et al. [2021] Muhammet Fatih Aslan, Muhammed Fahri Unlersen, Kadir Sabanci, and Akif Durdu.  CNN-based transfer learning–BiLSTM network: A novel approach for COVID-19 infection detection.  _Applied Soft Computing_ , 98(106912), 2021.  ISSN 15684946.  doi:10.1016/j.asoc.2020.106912.  URL https://doi.org/10.1016/j.asoc.2020.106912.
  * Zebin and Rezvy [2021] Tahmina Zebin and Shahadate Rezvy.  COVID-19 detection and disease progression visualization: Deep learning on chest X-rays for classification and coarse localization.  _Applied Intelligence_ , 51(2):1010–1021, 2021\.  ISSN 15737497.  doi:10.1007/s10489-020-01867-1.
  * Jain et al. [2020] Govardhan Jain, Deepti Mittal, Daksh Thakur, and Madhup K Mittal.  A deep learning approach to detect Covid-19 coronavirus with X-Ray images.  _Biocybernetics and biomedical engineering_ , 40(4):1391–1405, 10 2020.  ISSN 0208-5216.  doi:10.1016/j.bbe.2020.08.008.  URL http://www.ncbi.nlm.nih.gov/pubmed/32921862http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC7476608.
  * Mohammed et al. [2021] Mazin Abed Mohammed, Karrar Hameed Abdulkareem, Begonya Garcia-Zapirain, Salama A. Mostafa, Mashael S. Maashi, Alaa S. Al-Waisy, Mohammed Ahmed Subhi, Ammar Awad Mutlag, and Dac Nhuong Le.  A comprehensive investigation of machine learning feature extraction and classification methods for automated diagnosis of COVID-19 based on X-ray images.  _Computers, Materials and Continua_ , 66(3):3289–3310, 2021.  ISSN 15462226.  doi:10.32604/cmc.2021.012874.
  * Ismael and Şengür [2021] Aras M Ismael and Abdulkadir Şengür.  Deep learning approaches for COVID-19 detection based on chest X-ray images.  _Expert systems with applications_ , 164(114054), 2 2021\.  ISSN 0957-4174.  doi:10.1016/j.eswa.2020.114054.  URL http://www.ncbi.nlm.nih.gov/pubmed/33013005%****␣references.bbl␣Line␣975␣****http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC7521412.
  * Sahlol et al. [2020] Ahmed T. Sahlol, Dalia Yousri, Ahmed A. Ewees, Mohammed A.A. Al-qaness, Robertas Damasevicius, and Mohamed Abd Elaziz.  COVID-19 image classification using deep features and fractional-order marine predators algorithm.  _Scientific Reports_ , 10(1), 2020.  ISSN 20452322.  doi:10.1038/s41598-020-71294-2.  URL https://doi.org/10.1038/s41598-020-71294-2.
  * Yoo et al. [2020] Seung Hoon Yoo, Hui Geng, Tin Lok Chiu, Siu Ki Yu, Dae Chul Cho, Jin Heo, Min Sung Choi, Il Hyun Choi, Cong Cung Van, Nguen Viet Nhung, Byung Jun Min, and Ho Lee.  Deep Learning-Based Decision-Tree Classifier for COVID-19 Diagnosis From Chest X-ray Imaging.  _Frontiers in medicine_ , 7:427, 7 2020.  ISSN 2296-858X.  doi:10.3389/fmed.2020.00427.  URL http://www.ncbi.nlm.nih.gov/pubmed/32760732http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC7371960.
  * Mursalim and Kurniawan [2021] Muhammad Khaerul Naim Mursalim and Ade Kurniawan.  Multi-kernel CNN block-based detection for COVID-19 with imbalance dataset.  _International Journal of Electrical and Computer Engineering_ , 11(3):2467–2476, 2021.  ISSN 2088-8708.  doi:10.11591/ijece.v11i3.pp2467-2476.
  * Narayan Das et al. [2020] N. Narayan Das, N. Kumar, M. Kaur, V. Kumar, and D. Singh.  Automated Deep Transfer Learning-Based Approach for Detection of COVID-19 Infection in Chest X-rays.  _Ing Rech Biomed_ , PMID: 32837679; PMCID: PMC7333623., 2020.  ISSN 18760988.  doi:10.1016/j.irbm.2020.07.001.
  * Singh et al. [2020] Amitojdeep Singh, Sourya Sengupta, and Vasudevan Lakshminarayanan.  Explainable Deep Learning Models in Medical Image Analysis.  _Journal of Imaging_ , 6(6), 2020.  ISSN 2313-433X.  doi:10.3390/jimaging6060052.  URL https://www.mdpi.com/2313-433X/6/6/52.
  * Liang et al. [2021] Shuang Liang, Huixiang Liu, Yu Gu, Xiuhua Guo, Hongjun Li, Li Li, Zhiyuan Wu, Mengyang Liu, and Lixin Tao.  Fast automated detection of COVID-19 from medical images using convolutional neural networks.  _Communications Biology_ , 4(1):35, 2021.  ISSN 23993642.  doi:10.1038/s42003-020-01535-7.