# CURRENT AND FUTURE ROLE OF ARTIFICIAL INTELLIGENCE IN CARDIAC IMAGING

EDITED BY : Steffen Erhard Petersen, Karim Lekadir, Alistair A. Young and Tim Leiner PUBLISHED IN : Frontiers in Cardiovascular Medicine

#### Frontiers eBook Copyright Statement

The copyright in the text of individual articles in this eBook is the property of their respective authors or their respective institutions or funders. The copyright in graphics and images within each article may be subject to copyright of other parties. In both cases this is subject to a license granted to Frontiers. The compilation of articles constituting this eBook is the property of Frontiers.

Each article within this eBook, and the eBook itself, are published under the most recent version of the Creative Commons CC-BY licence. The version current at the date of publication of this eBook is CC-BY 4.0. If the CC-BY licence is updated, the licence granted by Frontiers is automatically updated to the new version.

When exercising any right under the CC-BY licence, Frontiers must be attributed as the original publisher of the article or eBook, as applicable.

Authors have the responsibility of ensuring that any graphics or other materials which are the property of others may be included in the CC-BY licence, but this should be checked before relying on the CC-BY licence to reproduce those materials. Any copyright notices relating to those materials must be complied with.

Copyright and source acknowledgement notices may not be removed and must be displayed in any copy, derivative work or partial copy which includes the elements in question.

All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary only. For further information please read Frontiers' Conditions for Website Use and Copyright Statement, and the applicable CC-BY licence.

ISSN 1664-8714 ISBN 978-2-88966-058-2 DOI 10.3389/978-2-88966-058-2

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office<EMAIL_ADDRESS>
# CURRENT AND FUTURE ROLE OF ARTIFICIAL INTELLIGENCE IN CARDIAC IMAGING

Topic Editors:

Steffen Erhard Petersen, Queen Mary University of London, United Kingdom Karim Lekadir, University of Barcelona, Spain Alistair A. Young, King's College London, United Kingdom Tim Leiner, University Medical Center Utrecht, Netherlands

Citation: Petersen, S. E., Lekadir, K., Young, A. A., Leiner, T., eds. (2020). Current and Future Role of Artificial Intelligence in Cardiac Imaging. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88966-058-2

# Table of Contents

*04 Editorial: Current and Future Role of Artificial Intelligence in Cardiac Imaging*

Karim Lekadir, Tim Leiner, Alistair A. Young and Steffen E. Petersen

*11 Machine Learning for Assessment of Coronary Artery Disease in Cardiac CT: A Survey*

Nils Hampe, Jelmer M. Wolterink, Sanne G. M. van Velzen, Tim Leiner and Ivana Išgum

*19 Machine Learning Approaches for Myocardial Motion and Deformation Analysis*

Nicolas Duchateau, Andrew P. King and Mathieu De Craene

Aurélien Bustin, Niccolo Fuin, René M. Botnar and Claudia Prieto

Matthew E. Fenech and Olly Buston

Kathleen Gilbert, Charlène Mauger, Alistair A. Young and Avan Suinesiaputra

# Editorial: Current and Future Role of Artificial Intelligence in Cardiac Imaging

Karim Lekadir 1 \*, Tim Leiner 2 , Alistair A. Young3 and Steffen E. Petersen4,5

1 Universitat de Barcelona, Artificial Intelligence in Medicine Lab (BCN-AIM), Departament de Matemàtiques and Informàtica, Barcelona, Spain, 2 Department of Radiology, Utrecht University Medical Centre, Utrecht, Netherlands, 3 School of Biomedical Engineering & Imaging Sciences, King's College London, London, United Kingdom, 4 Barts Heart Centre, Barts Health NHS Trust, London, United Kingdom, 5 NIHR Barts Biomedical Research Centre, William Harvey Research Institute, Queen Mary University of London, London, United Kingdom

Keywords: artificial intelligence, cardiac imaging modalities, big data, cardiac image analysis, cardiovascular personalized medicine, AI adoption and translation

**Editorial on the Research Topic**

**Current and Future Role of Artificial Intelligence in Cardiac Imaging**

### INTRODUCTION

Cardiovascular disease is currently the most common cause of morbidity and mortality worldwide (1) and thus remains an important focus for both biomedical and technological research. In the age of personalized medicine, cardiac imaging is expected to play an important role to enable more accurate and advanced quantification of structural and functional changes due to cardiovascular disorders. However, despite advances in cardiac imaging modalities, such as echocardiography, cardiovascular magnetic resonance, cardiac computed tomography, and nuclear cardiology, the heart remains a challenging anatomical organ to image and assess compared to other organ systems. The main challenges faced by cardiac imaging include the perpetual cardiac and respiratory motions, the complex geometry of the ventricles, atria and arteries, the oblique orientation of the heart with respect to the body, and the small size of some of the cardiac structures, such as the coronary arteries, trabeculae and papillary muscles, as well as the large variability in imaging conditions and protocols (including non-contrasted and contrast-enhanced cardiac imaging sequences).

Consequently, advanced tools are needed to optimize the use of cardiac imaging and to support clinicians throughout the whole value-chain of cardiovascular practice, including improved image acquisition, automated cardiac quantification, cardiac tissue characterization, imaging biomarker discovery, and clinical decision support. In this context, artificial intelligence (AI), including machine learning and computer vision, has emerged as one of the most promising topics over the last 5 years. Combined with the exponential increase in computing power, AI provides unprecedented opportunities to leverage the available collections of cardiac imaging data for developing more robust cardiac image analysis algorithms, to uncover currently unknown clinical knowledge on cardiac health and disease, and to build novel software tools that will impact clinical cardiology. This area is expected to benefit from the current efforts to provide access to largescale and high-quality image data for the scientific community. In the US, for example, existing studies such as the Multi-Ethnic Study of Atherosclerosis and the Framingham Heart Study have for a long time compiled thousands of cardiac images. More recently, in the United Kingdom, the UK Biobank has been acquiring cardiac MRI images from tens of thousands of individuals (2).

### Edited and reviewed by:

Hendrik Tevaearai Stahel, Bern University Hospital, Switzerland

> \*Correspondence: Karim Lekadir<EMAIL_ADDRESS>
#### Specialty section:

This article was submitted to Cardiovascular Imaging, a section of the journal Frontiers in Cardiovascular Medicine

> Received: 20 May 2020 Accepted: 30 June 2020 Published: 07 August 2020

#### Citation:

Lekadir K, Leiner T, Young AA and Petersen SE (2020) Editorial: Current and Future Role of Artificial Intelligence in Cardiac Imaging. Front. Cardiovasc. Med. 7:137. doi: 10.3389/fcvm.2020.00137 In Europe, the euCanSHare project funded by the European Commission is developing a data sharing and analytics platform to facilitate access to large-scale cardiac imaging and nonimaging data from multiple centers (www.eucanshare.eu). Similar initiatives and large cohorts are expected to emerge across the globe in the years to come, which will further enhance the potential of AI in cardiac imaging and clinical cardiology.

To promote and guide further research and developments in AI for cardiac imaging, several experts and leading institutions in the field have recently published position and review papers that outline the initial achievements, discuss the current challenges and identify future perspectives. For example, Dey et al. summarized the most promising AI methods for cardiac imaging by distinguishing between the use of classical AI and advanced approaches (3). Al'Aref et al. reviewed some clinical applications of AI in cardiac imaging by considering each cardiac imaging modality separately (4). Litjens et al. focused on the sole application of deep learning methods for cardiac image analysis (5). Finally, Petersen et al. outlined the current challenges and emerging opportunities (6), emphasizing the importance of addressing non-technical aspects of AI in cardiac imaging such as patient acceptance, data protection and AI regulation.

While these papers provided an overall presentation and promotion of the field, the goal of this special issue entitled "current and future role of AI in cardiac imaging" is to compile more detailed and focused reviews covering the whole valuechain of AI in cardiac imaging. Specifically, we have invited active experts across the globe to submit in-depth reviews on several keys areas of AI in cardiac imaging, including (1) cardiac image reconstruction, (2) cardiac image segmentation, (3) cardiac shape and motion analysis, (4) computer-aided diagnosis, (5) imaginggenetics integration, and (6) socio-ethical impact and regulations. This comprehensive special issue, totaling nearly 1,000 references from the field, will constitute an unprecedented resource for researchers, both novice and experienced, to study in detail the methods, applications, strategies, datasets, tools, hypotheses, limitations and opportunities that are of direct relevance to each aspect of AI in cardiac imaging. Importantly, for each paper, we requested the authors to provide descriptions and discussions for both AI and clinical audiences, to enhance the democratization and promotion of AI in cardiac imaging, and thus future collaborations and developments in the field.

### SPECIAL ISSUE CONTENT

This special issue cover six specific areas and application domains of AI in cardiac imaging, as shown in **Figure 1** and presented as follows:

### Cardiac Image Reconstruction

The first paper of this special issue focuses on the very first step of the cardiac imaging workflow, i.e., enhancing cardiac image acquisition using AI. Fast and portable cardiac imaging such as echocardiography inherently suffers from low image quality, while high resolution cardiac imaging such as CMR requires long acquisition times to address the cardiac and respiratory motions, as well as the need to highlight the different types of cardiac structures, tissues and vessels. For a long time, enhancing and accelerating image acquisition for modalities such as CMR was targeted by developing new CMR imaging/physics sequences and techniques, such as efficient pulse sequences, motion compensation techniques, multiple radio-frequency receiver coils for parallel imaging and compressed sensing. In this special issue, Bustin et al. thoroughly surveyed emerging AI techniques for accelerating and enhancing CMR image reconstruction. Concretely, AI provides a unique opportunity to perform CMR acquisition using undersampling strategies that acquire less image data than needed, followed by learning-based estimation of the sparse domain from existing data. The authors reviewed first the initial learning-based techniques based on dictionaries of transforms (from low to high resolution domains) learned from the acquired under-sampled data itself. Subsequently, they focused their attention on recent deep learning-based approaches, which learn the reconstruction transforms from low-resolution to high-resolution images offline based on training data. They surveyed in detail the many advances over the last 2 years, describing recent applications to both 2D dynamic cardiac imaging and 3D whole-heart CMR imaging. As deep learning based CMR reconstruction is novel as well as popular, they concluded their review by discussing the future avenues to improve real-world validation, including assessment of reconstruction quality and generalization.

### Cardiac Image Segmentation

In the next review papers, this special issue addressed the next stage of cardiac image analysis, namely cardiac image segmentation, which is by far the most covered topic by AI researchers in cardiac imaging. Deep learning approaches have been particularly popular as they have shown to generate highly accurate results when trained on large manually segmented data. At the same time, clinicians have been highly receptive to such automated black-box methods as they accelerate their clinical work without interfering in the decision making. In this special issue, Chen et al. put together a comprehensive review of deep learning techniques for cardiac image segmentation, totaling over 100 papers describing applications to various imaging modalities (echocardiography, MRI, and cardiac CT) and to the main cardiac structures (ventricle, atria, and vessels). They concluded that there is no universally optimal deep learning implementation for cardiac image segmentation, and suggested that algorithms need to be customized and optimized for each application depending on the imaging modality, protocol (e.g. contrast vs. non-contrast), and cardiac structure (left vs. right ventricle). For the immediate future, they discussed the importance of developing segmentation methods that can generalize well across various imaging modalities, scanners, and pathologies. Finally, in order to encourage reproducible research, the authors provided a summary of public datasets for training and testing new deep learning models, as well as public code repositories that include recently developed techniques.

In a more focused review, Jamart et al. surveyed one of the most challenging cardiac image segmentation applications, namely atrial segmentation from late enhanced cardiac MRI. This contrast enhanced imaging sequence is the technique of

choice in clinical practice to quantify fibrosis and assess atrial fibrillation. However, the task is complicated by the geometrical complexity and small size of the atrial chambers, which are also constrained by thin walls. Moreover, the anatomical boundaries on the late enhanced MRI often lack clear contrast, which can further mislead the segmentation algorithms. As a result, until the advent of deep learning, very few techniques had been attempted for automated segmentation of the atria. In this review, the authors summarized the recent deep learning developments in this field, including multi-stage and multi-scale conventional neural networks to address the image class imbalance that is inherent to atrial segmentation (the atrial cavity represents only a small fraction of the image volume), as well as to provide contextual cues to better differentiate the atrial boundaries from the surrounding structures. The best reported performance in the survey reached a 93.2% segmentation accuracy, which is highly promising given the complexity of the task. Arguably, the most important future work in this domain is the automated detection of the fibrosis using advanced AI approaches, which will represent an important Research Topic in the years to come.

### Cardiac Shape and Motion Analysis

Image segmentation of the cardiac boundaries as described above is a pre-requisite in clinical practice to estimate standard clinical indices such as chamber volumes and ejection fraction for cardiac assessment. However, existing research has shown that more detailed information about cardiac geometry and regional motion is expected to improve future clinical assessment of normal and abnormal cardiac (dys)function. In this special issue, Gilbert et al. presented a review of the so-called statistical atlases of cardiac anatomy, which are widely used to model complex shape and function variability across subpopulations, and thus to extract new descriptors of cardiac geometry of relevance to cardiac health and disease. In this paper, the authors firstly describe the existing and newer methods for building clinically useful statistical cardiac atlases from annotated cardiac contours. Subsequently, the paper discussed that, powered by supervised or unsupervised machine learning algorithms, statistical cardiac shape analysis can be used to automatically identify and quantify abnormal shape deviations, and to provide morphometric indices that are optimally associated with clinical factors.

In another review, Duchateau et al. described in great depth the literature on cardiac motion quantification and analysis based on machine learning. With these techniques, the idea is to learn new advanced representations and patterns of myocardial motion and deformation (displacement, velocity, deformation, torsion, strain) from representative samples of cardiac images, such as echocardiography or MRI. In particular, two families of approaches were reviewed, namely (1) traditional techniques that apply machine learning onto explicit features of myocardial motion/deformation (displacement fields calculated from the images), and (2) more recent approaches based on neural networks applied directly to the image data to extract and analyze new spatiotemporal signatures from local image patches around the myocardium. In this paper, the authors described the entire workflow of steps and methods required to derive physiologically meaningful and clinically useful analyses of cardiac motion. Finally, they discussed the next steps toward clinical adoption of machine learning based cardiac motion quantification, including community benchmarking, standardization initiatives, and clinical interpretability of the extracted spatiotemporal signatures for abnormality localization.

### Computer-Aided Diagnosis

Generally, the main AI developments in cardiac imaging have mostly addressed cardiac image analysis tasks before clinical decision making, with the aim to enhance cardiac image acquisition, facilitate cardiac image segmentation, and estimate advanced indices of cardiac shape and function. However, AI is also expected to impact clinical decision making in the future, such as to enable earlier and more precise diagnosis, as well as treatment planning and response estimation. To illustrate this, this special issue includes two review papers centered on cardiac diagnosis. First, Martin-Isla et al. surveyed in detail the area of image-based cardiac diagnosis using machine learning. This included a step-by-step description of the techniques for building and validating new AI models of cardiac diagnosis. The authors also described emerging techniques for more precise diagnosis, including radiomics (omics for radiology) (7). The authors then reviewed more than 100 papers on AI- and imagedriven diagnosis of coronary heart disease, cardiomyopathy, heart failure, and valve disease. Interestingly, the survey showed that some complex cardiac diseases are yet to be extensively investigated by the AI community, such as atrial fibrillation. Finally, the paper discussed current obstacles that limit the applicability of AI-driven diagnosis in cardiac imaging, in particular the lack of interpretability, which must be addressed in the years to come to enable clinicians to understand and trust the AI-generated diagnoses and decisions.

In a second review paper on AI for cardiac diagnosis, Hampe et al. focused on the assessment of coronary artery disease from non-invasive CT using machine learning. In clinical practice, catheter-guided X-ray angiography and intravascular ultrasound provide detailed information on coronary stenosis and plaque composition, but they are limited by their invasive nature. CT imaging provides a promising alternative but offers reduced contrast between the atherosclerotic plaque constituents, and thus AI is expected to play a role for the extraction of detailed and clinically useful information from the non-invasive images. This review surveyed classical and modern machine learning methods to estimate coronary stenosis, to discriminate between calcified, non-calcified and mixed plaques in CT, and to characterize fibrous and lipidic plaque constituents. Furthermore, the paper described recent methods, in particular deep learning based, for predicting fractional flow reserve directly from CT, by learning the relationship between CT features and fractional flow reserve based on a training sample of corresponding CT and invasive imaging. In their discussion, the authors noted an important limitation of current models, which are based on small training samples, as images with manually characterized plaques are more difficult to obtain than, for example, manual annotations of cardiac boundaries. Thus, this field of AI in cardiac imaging is expected to further develop for future clinical use as additional and larger datasets become available in the years to come.

### AI Integration With Non-imaging Data

To realize the promise of precision medicine in cardiology, cardiac imaging is a central piece of the puzzle. However, non-imaging data play an important role, in particular omics and health data, as they allow to build multi-scale AI models that integrate patient-specific biomolecular, phenotypic, environmental, and clinical information. Such integrated AI models are expected to lead to improved diagnosis and treatment selection, as well as to better clinical outcomes. This special issue included an interesting review by de Marvao et al. of AI-driven integrated cardiac imaging-genetics studies, which aim to characterize the complex interplay between cardiac imaging phenotypes, environmental and genetic factors. Several concrete examples of AI-empowered imaging-genetics studies were provided, such as to enable more stratified diagnosis of heart failure, predict treatment response in cardiomyopathic patients, identify genetic variants or proteomic signatures of high-risk atherosclerotic plaques, or predict positive cardiac remodeling after cardiac resynchronization therapy. While the use of AI in cardiovascular imaging-genetics is shown to have great potential, the review noted that the challenges of AI in genetics and imaging separately are amplified by combining these very large data. Thus, further research is expected to address more ambitious wholegenome and high-resolution whole-heart imaging studies, and to derive multi-scale AI solutions for clinical practice integrating imaging, biological and clinical data.

### Ethical, Social, and Political Issues

While the review papers described above dealt with technical and clinical aspects of AI in cardiac imaging, this special issue concludes with a paper by Fenech and Buston that reviews the ethical, social, and political issues that are being investigated to facilitate future acceptance and deployment of the AI solutions in cardiac imaging. These include clarifying the impact of AI solutions on the roles of cardiologists, radiologists, and other doctors, future liability of clinicians vs. AI manufacturers, as well as on the altered relationships between healthcare professionals, patients, their relatives, and administrators. Data sharing and privacy issues were also are reviewed in the paper, focusing on the challenges to manage patient informed consent for AI solutions that remain difficult to understand (and trust) by the general public and clinicians alike. From a social point of view, initial studies reviewed in this paper suggest that there is a concern that AI solutions may remain biased and in fact exacerbate health inequalities. Furthermore, the authors discussed the need to include patients and citizens in the AI development process, to take into close consideration their requirements, expectations, and behaviors. Other important issues, such as algorithmic transparency, fairness, and regulation, were also discussed at length. For addressing these key issues and to optimize adoption by clinicians, patients, and regulators, the paper emphasized the importance of developing principles and translating them into policies in the years to come. In the general context of AI, imaging and cardiology in particular are expected to play an important role, as exemplified by the fact that they have been the healthcare domains with the greatest number of FDA approvals for novel data-driven technologies in the recent years.

## FUTURE PERSPECTIVES

This special issue reviewed AI developments and opportunities for each task of the cardiac imaging clinical workflow, surveying in detail cardiac imaging acquisition, segmentation and quantification, clinical decision support and precision cardiovascular medicine through integration with genomics data, in addition to ethical, social, and regulatory aspects. A close look at the statistics from **Figures 2**, **3**, gathered from the publications reviewed in this special issue, shows a continuous increase in the research output in AI for cardiac imaging over the last 5 years. Interestingly, while cardiac image segmentation has received the most attention in the field as shown in **Figure 3**, due to the urgent need to accelerate the contouring process, it is also the only cardiac imaging task for which there is a decrease in the number of publications between 2018 and 2019 (**Figure 2**). This may be explained by the advances made possible by deep learning in the field, combined with an increasing interest from the community to invest in other AI applications such as cardiac image reconstruction or computer-aided diagnosis.

While this special issue described six main AI applications in cardiac imaging separately, we believe the ultimate aim should

FIGURE 2 | Number of papers reviewed in the six categories of cardiac imaging tasks in the last 5 years (2015 to 2019), showing an increase in the research output continuously and for all tasks, except for segmentation which decreased between 2018 and 2019 (over 550 papers reviewed in total). The subfigures are taken from the different papers in this special issue.

be to integrate these separate tasks into one single smooth, efficient and user-friendly pipeline for clinical cardiologists. As an example, this multi-task integration is the scope of the UK-based research project "SmartHeart: Next-generation cardiovascular healthcare via integrated image acquisition, reconstruction, analysis and learning" funded by UK's Engineering and Physical Sciences Research Council.

Additionally, while the special issue covered all important steps of cardiac imaging, it is worth mentioning other research tasks, at their very initial stages of development, which have not yet been covered in this special issue. For example, "automated quality control" is expected to enhance the cardiac image analysis workflow as large volumes of research and clinical data become available, and as the demand for AI-driven automation and robustness will increase in clinical practice. Here, it is worth listing a few preliminary works, such as automated quality control of CMR images using a deep learning approach to identify suboptimal image contrast or heart coverage (8). Other works have instead focused on quality control of the final image segmentation results using classical AI (9) or neural networks (10). Another area that may benefit from AI is "image-based computational cardiology," which builds patient-specific digital models of the heart to simulate treatment response. While this area has been traditionally addressed using pure physiological and mechanistic models, researchers are now investigating the integration of machine learning to improve the accuracy and speed of the personalized simulated outputs (11). Furthermore, as larger datasets become available, it is expected that predictive models of disease progression will be developed and validated, including by integrating imaging with non-imaging predictors (e.g., socio-demographic, biomarker, lifestyle and genomic data). Finally, while this special issue is dominated by AI applications in echocardiology, CMR and cardiac CT, there have also been machine learning applications in "nuclear cardiology" (12), and these are expected to increase in the years to come as larger nuclear medicine datasets (both PET and SPECT) become available.

To conclude this editorial, we wish to emphasize the need in the next years for more concerted efforts dedicated to enhancing the technical and clinical advances described in this special issue, but also to address non-technical and nonclinical aspects of AI in cardiac imaging. Importantly, there is a need for community-defined standards and guidelines for validating and adopting future AI solutions, including metrics and procedures to evaluate performance, bias and errors, clinical effectiveness, degree of interpretability, and even cost-effectiveness. Benchmarking datasets and tools are also required to enable transparent and comparative analysis of the AI solutions across research institutions and players. For example, a test-retest reference dataset was recently compiled to assess reproducibility of machine learning CMR studies (13), while an international challenge on multi-center and multi-vendor cardiac imaging segmentation was organized to test generalizability across scanners (Siemens, Philips, General Electric and Canon)1 . Finally, ethical and regulatory aspects will need to be established in a multi-stakeholder collaboration between experts in AI, bioethics and cardiac imaging, but also with the involvement of patient associations, private companies, and public authorities.

1https://www.ub.edu/mnms/

### REFERENCES

### AUTHOR CONTRIBUTIONS

All co-authors discussed the structure and content of the special issue and paper. KL wrote the first draft. TL, AY, and SP revised the manuscript.

### FUNDING

The work of KL was partly funded by the European Union's Horizon 2020 research and innovation programme under grant agreement no. 825903 (euCanSHare project), and by the Ramon y Cajal Programme of the the Spanish Ministry of Economy and Competitiveness under grant no. RYC-2015-17183.

**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2020 Lekadir, Leiner, Young and Petersen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Machine Learning for Assessment of Coronary Artery Disease in Cardiac CT: A Survey

Nils Hampe1,2,3 \*, Jelmer M. Wolterink 1,2,3, Sanne G. M. van Velzen3 , Tim Leiner 4 and Ivana Išgum1,2,3,5

1 Department of Biomedical Engineering and Physics, Amsterdam University Medical Center, University of Amsterdam, Amsterdam, Netherlands, 2 Amsterdam Cardiovascular Sciences, Amsterdam University Medical Center, University of Amsterdam, Amsterdam, Netherlands, 3 Image Sciences Institute, University Medical Center Utrecht, Utrecht, Netherlands, 4 Department of Radiology, University Medical Center Utrecht, Utrecht, Netherlands, 5 Department of Radiology and Nuclear Medicine, Amsterdam University Medical Center, University of Amsterdam, Amsterdam, Netherlands

#### Edited by:

Fabrizio Ricci, G. d'Annunzio University of Chieti and Pescara, Italy

#### Reviewed by:

Maria A. Zuluaga, Institut Eurécom, France John Hoe, MediRad Associates Ltd, Singapore

\*Correspondence:

Nils Hampe<EMAIL_ADDRESS>
#### Specialty section:

This article was submitted to Cardiovascular Imaging, a section of the journal Frontiers in Cardiovascular Medicine

Received: 30 September 2019 Accepted: 12 November 2019 Published: 26 November 2019

#### Citation:

Hampe N, Wolterink JM, van Velzen SGM, Leiner T and Išgum I (2019) Machine Learning for Assessment of Coronary Artery Disease in Cardiac CT: A Survey. Front. Cardiovasc. Med. 6:172. doi: 10.3389/fcvm.2019.00172

Cardiac computed tomography (CT) allows rapid visualization of the heart and coronary arteries with high spatial resolution. However, analysis of cardiac CT scans for manifestation of coronary artery disease is time-consuming and challenging. Machine learning (ML) approaches have the potential to address these challenges with high accuracy and consistent performance. In this mini review, we present a survey of the literature on ML-based analysis of coronary artery disease in cardiac CT. We summarize ML methods for detection and characterization of atherosclerotic plaque as well as anatomically and functionally significant coronary artery stenosis.

Keywords: machine learning, coronary artery disease, atherosclerotic plaque, coronary artery stenosis, cardiac CT

### 1. INTRODUCTION

Diagnosis and monitoring of coronary artery disease (CAD) is increasingly based on non-invasive imaging with computed tomography (CT), allowing excellent visualization of the coronary arteries with high spatial resolution. Cardiac CT exams consist of hundreds of slices and the number of cardiac CT studies has been steadily increasing (1). This has led to an increased workload for medical professionals, which in combination with shortages of trained cardiac imagers (2) might lead to cardiac CT underuse in the clinic. Machine learning (ML) could offer a way to address these challenges and facilitate automatic cardiac CT analysis with consistent and accurate results. Furthermore, ML algorithms might enable an increased range of secondary diagnoses.

This survey provides an overview of ML algorithms for detection, characterization, and quantification of CAD in cardiac CT. We searched PubMed for articles related to ML-based assessment of CAD in cardiac CT published within the last 10 years (search strategy in **Supplementary Materials**) which led to inclusion of 59 studies. The structure of this survey is as follows. We provide a brief primer on ML in section 2. Applications of ML for automatic detection and characterization of atherosclerotic plaque are summarized in section 3. Studies focusing on ML for anatomical and functional evaluation of luminal stenosis are summarized in section 4. Finally, section 5 provides a discussion of outstanding challenges for transfer of ML algorithms into the clinic.

### 2. MACHINE LEARNING

Machine learning comes in many flavors, but most applications in cardiac CT use supervised learning. In supervised learning, a model is optimized to provide the correct labels as defined by the reference standard during training, and predict a label to new and unseen samples during testing.

Each sample can be described based on characteristics or features. Among the simplest ML algorithms are k-nearest neighbor (kNN) classifiers, which look for training samples with similar feature values to a test sample, and assign the test sample to the majority class among these training samples. Linear classifier (LC) models like support vector machines (SVM) aim to find a linear combination of features to separate samples in different classes. Alternatively, samples can be separated by thresholding feature values along a single axis. This is unlikely to lead to highly accurate classifiers, but by consecutively applying thresholds, a decision tree model can be built for more accurate classification.

ML performance can often be improved by combining predictions of multiple models. Ensembles (E) combine predictions of multiple simultaneously executed models, e.g., by averaging predictions of decision trees in a random forest (RF). In boosting (BO), models are applied consecutively and each model is trained to correct errors of its predecessors. Finally, artificial neural networks (ANNs) transform samples into targets through layers of trainable neurons, which are loosely based on biological neurons. While ANNs have been around since the 1950s, it has recently become possible to train networks that have many layers, i.e., deep learning. The success of deep learning in medical image analysis has been to a large extent due to the inclusion of trainable image filters in so-called convolutional neural networks (CNNs), which can be trained to extract valuable features from raw image data (3). For a more in-depth introduction to ML and deep learning, please refer to Jordan and Mitchell (4).

### 3. ATHEROSCLEROTIC PLAQUE DETECTION, CHARACTERIZATION, AND QUANTIFICATION

CT offers a non-invasive alternative to e.g., catheter-guided X-ray angiography, optical coherence tomography, and intravascular ultrasound (IVUS) for atherosclerotic plaque visualization. Characterization and quantification of plaque in CT provide insight in different stages of CAD (5). In this section, we survey analysis of methods for calcified plaque (section 3.1) and noncalcified and mixed plaque (section 3.2). Reviewed papers are listed in **Table 1**.

### 3.1. Calcified Plaque

Coronary artery calcification (CAC) quantification or scoring is typically performed in dedicated non-contrast-enhanced, ECGtriggered, calcium scoring CT images (CSCT). Using dedicated software, an expert identifies voxels with a density over 130 Hounsfield units (HU) in the coronary arteries. Identified CAC is then quantified according to its volume, density, or a TABLE 1 | Publications related to analysis of (A) calcified and (B) non-calcified and mixed atherosclerotic plaque.

(B) Non-calcified plaque Detect Characterize Training Testing Classifier

Check marks in (A) indicate detection (Detect) or characterization (Characterize) of plaque, check marks in (B) indicate analysis in dedicated non-contrast-enhanced calcium scoring CT (CSCT), chest CT (Chest CT) or coronary CT angiography (CCTA) images. The number of patients included for method development (Training) and evaluation (Testing) are listed, \* indicates cross-validation and - indicates training on non-patient data. The classifier with which the primary result was obtained is indicated (Classifier, see section 2 for abbreviations).

† A total of 15 scans was divided into training sets ranging from 1 to 13 and respective test sets comprised of the remaining scans.

combination of both (27). CAC cannot only be quantified in CSCT, but also in other kinds of CT images visualizing the heart, such as cardiac CT angiography (CCTA) and non-gated chest CT. Calcium scoring is not considered a difficult task for trained clinicians, but it is time-consuming when performed in large numbers of images. Hence, automatic ML-based methods have been proposed.

ML-based calcium scoring methods proposed prior to the advent of deep learning have focused on identification of CAC lesions among a large set of samples, i.e., groups of connected voxels above 130 HU. Samples are described with features such as size, shape, appearance and location to distinguish CAC from other candidate lesions such as calcifications in the aorta. Location features are of particular importance, as recognized by Liu et al. (15), Kurkure et al. (16), and Brunner et al. (17) who proposed a heart coordinate system. Similarly, Sánchez et al. (14) described candidate locations relative to anatomical landmarks. Išgum et al. (13) used multi-atlas registration to estimate the location of the coronary artery tree, while Shahzad et al. (12) and Wolterink et al. (11) estimated the location of three major coronary arteries for per-vessel calcium scoring. Yang et al. (9) extracted coronary artery centerlines in CCTA images and propagated these to CSCT images of the same patients to provide location features.

Deep learning-based methods have typically classified individual voxels instead of candidate lesions. Due to the extreme imbalance between numbers of CAC and background voxels in CT images, Wolterink et al. (10) proposed to use two CNNs, where one CNN identified candidate voxels in CCTA and the second CNN further discriminated among identified candidates. Similarly, Lessmann et al. (8) used two CNNs to identify calcified voxels in chest CT. Cano-Espinosa et al. (7) and de Vos et al. (6) avoid voxel-based classification altogether by directly regressing calcium scores in chest CT, enabling automatic scoring in less than a second.

Automatic CAC scoring methods have been validated in large data sets (28) and in other types of CT scans in which the heart is routinely visualized, such as attenuation correction images for PET-CT (29) and CT images acquired for radiotherapy treatment planning (30–32). Wolterink et al. presented a public data set with reference standard for standardized evaluation of CAC scoring in CSCT (33).

### 3.2. Non-calcified Plaque

Non-calcified plaque is typically lipid-rich and vulnerable to rupture, causing acute coronary syndrome (34). ML-based analysis methods in CCTA have been developed for detection or localization of non-calcified plaque, as well as characterization of lipid and fibrous plaque components.

Coronary artery localization by means of centerline extraction is a typical preprocessing step for ML-based plaque analysis. Traditionally, many automatic centerline extraction methods have been based on minimum cost paths between proximal and distal artery points (35, 36). ML has been used to verify automatic centerline extraction results with an RF (25) or CNN (37). Alternatively, centerlines can be iteratively extracted based on a single seed point. Wolterink et al. (38) showed how such a tracker can be guided by a 3D CNN that locally detects the artery orientation.

Coronary artery centerlines can be used to reconstruct CCTA volumes into images that allow better plaque visualization and identification. Zhao et al. (21), Jawaid et al. (22), Wei et al. (23), and Zuluaga et al. (26) used cross-sectional images along the coronary artery centerline to extract features describing the vessel wall shape and texture. In Jawaid et al. (22) and Wei et al. (23), these features were used in an SVM or linear classifier to determine whether the image contained non-calcified plaque. Similarly, Zuluaga et al. (26) used such features to train an SVM classifying lesion segments as either healthy or diseased, i.e., containing non-calcified or calcified plaque. Zhao et al. (21) trained an SVM to classify cross-sectional images as healthy or containing non-calcified, calcified, or mixed plaque. For the same task, Zreik et al. (20) trained a recurrent CNN that did not depend on hand-crafted feature extraction. Kelm et al. (25) used an RF classifier to classify whether noncalcified or calcified plaque was present along a coronary artery centerline segment.

Characterization of individual components in non-calcified plaque is a challenging task due to low-contrast boundaries between plaque components (39). Yamak et al. (24) exploited additional attenuation data provided by dual-energy CT to characterize plaque in manually determined regions of interest in axial slices. To validate their model in patient scans, manual CCTA annotations by an expert were used. However, obtaining reliable manual reference annotations for non-calcified plaque in CCTA is challenging. Kolossváry et al. (18) determined the reference standard in CCTA through registration of histology images to ex-vivo CCTA scans. Features were extracted for each cross-sectional image and lesions were classified into advanced or early stage atherosclerosis using a linear classifier. Alternatively, Masuda et al. (19) used an in-vivo IVUSbased reference standard to train a boosting classifier with histogram-based features distinguishing fibrous from lipid plaque in CCTA.

### 4. CORONARY STENOSIS DETECTION AND CHARACTERIZATION

Non-invasive assessment of CAD-induced stenotic lesions in CT prior to invasive treatment may prevent unnecessary costs and complications (40). Therefore, CT images have long been used to assess the anatomical significance of lesions by a local measurement of luminal narrowing. However, determination of the functional significance of a lesion by taking physiology into account can better stratify patients in need of treatment (41). In this section, we review ML algorithms for the detection and quantification of anatomically (section 4.1) and functionally (section 4.2) significant stenosis. Reviewed papers are listed in **Table 2**.

### 4.1. Anatomical Significance

Identification of anatomically significant stenotic lesions in CCTA, i.e., those lesions causing a luminal narrowing of at least 50%, allows a first assessment of the severity of stenosis in patients with symptoms of CAD. While this assessment is often based on visual estimation by a clinician, this is a difficult task (56) with substantial inter-observer variability (57). ML-based automatic approaches could reduce this variability.

Stenosis detection typically requires a local measurement of the lumen diameter and an estimation of the healthy lumen diameter. These estimates can be based on automatically extracted centerlines (section 3.2). Many centerline extraction methods also estimate the luminal radius at each centerline point, assuming a circular coronary artery profile (25, 38). However, circular artery profiles are not a realistic assumption for diseased vessel segments. Automatically extracted centerlines can also be used as an initialization for more detailed lumen segmentation. Huang et al. (44) used centerlines to obtain a reformatted image in which the lumen was segmented using a 3D CNN. Lee et al. (42) use centerlines to TABLE 2 | Publications related to (A) anatomically and (B) functionally significant stenosis detection.

Check marks indicate arterial (Artery) or myocardial (Myocardium) analysis. The number of patients included for method development (Training) and evaluation (Testing) are listed, \*indicates cross-validation and - indicates training on non-patient data. The classifier with which the primary result was obtained is indicated (Classifier, see section 2 for abbreviations).

obtain a tube-shaped prior that is deformed to segment the coronary lumen.

Lumen segmentation is often considered a preprocessing step for stenosis detection, but it has been shown that stenosis degree can also be directly determined based on image data. Zuluaga et al. (48) detected stenosis and artery bifurcations with an SVM based on features obtained from concentric circles in cross-sectional images. Similarly, Kang et al. (45) used geometrical and plaque features in an SVM to detect obstructive lesions (> 50% narrowing) and non-obstructive lesions (25–50% narrowing). Zreik et al. (20) used a recurrent CNN to detect anatomically significant stenosis along the centerline. Freiman et al. (43) detected stenosis of at least intermediate severity (> 40% narrowing) using deep sparse autoencoders, a variation on CNNs.

Coronary stenoses are located in the arteries, but may restrict blood flow to myocardial segments. Mukhopadhyay et al. (47) used an ML approach to identify myocardial segments (58) affected by coronary stenosis. Hand-crafted feature vectors describing the endocardial surface shape were combined using a bag-of-words approach and classified with an ANN to identify affected segments. Xiong et al. (46) performed analysis of the full myocardium to detect existence of at least one anatomically significant stenosis. Instead of the shape of the endocardial surface, features in this approach described the attenuation and wall thickness of myocardial segments.

### 4.2. Functional Significance

The sensitivity of CCTA-based anatomical stenosis evaluation for detection of functionally significant stenosis is high when evaluated visually, but its specificity is moderate (41). The current reference standard for determination of functional significance of a stenosis is given by its fractional flow reserve (FFR), i.e., the ratio of flow distal of the stenosis to the flow proximal of the stenosis. FFR is measured invasively by inserting a special catheter in the coronary artery under hyperemic conditions. FFR below 0.80 indicates need for intervention (59). Treatment based on invasive FFR measurements can improve patient outcomes (59), but measurement of FFR is still relatively uncommon, which is due to associated cost and risk, as well as lack of vasodilator drugs (60).

FFR estimation based on CCTA scans (FFRCT) could provide reproducible physical measurement without the drawbacks of invasive procedures. FFRCT has traditionally been based on computational fluid dynamics (CFD) (61, 62), i.e., numerical simulation of blood flow in a coronary tree model extracted from CCTA using lumen segmentation methods (section 4.1). These methods are accurate (63) but computationally expensive due to their iterative nature. This precludes their deployment on local workstations, and instead CFD simulations are typically performed on off-site dedicated systems. ML could be used to significantly speed up estimation of FFRCT.

Itu et al. (55) proposed an ANN model to predict an FFR value for each segment in the coronary artery tree, given local features based on the segment's geometry and global features based on the most severe stenoses. To train this model, a large data set of 12,000 synthetic coronary artery trees was generated and a reference standard was obtained through conventional CFD simulation. By only performing CFD simulations once in a training phase, the time required to perform FFRCT was reduced by two orders of magnitude. The diagnostic value of this method has been demonstrated thoroughly (64–76). Yu et al. (77) further demonstrated additional prognostic value of CT morphological index for the method proposed by Itu et al. (55). Wang et al. (50) proposed to use a recurrent ANN that can model long-range dependencies between segments.

Both conventional CFD-based FFRCT and the methods proposed in Wang et al. (50) and Itu et al. (55) are based only on the geometry of the coronary artery tree model, and are thus susceptible to errors by the segmentation method used to obtain this model (78). Instead, Dey et al. (52) proposed to combine geometric features with semi-automatically obtained plaque and attenuation gradient measurements to identify arteries with functionally significant stenosis. Other methods skip explicit coronary artery centerline extraction and lumen segmentation altogether. Kumamaru et al. (49) trained a CNN to extract a map showing the contrast-enhanced territories in CCTA and used this map in a classifier to predict the minimum FFR value in a patient. Alternatively, analysis can be moved from the cause the coronary arteries—to the effect, i.e., the myocardium. Han et al. (54) subdivided the separated endocardium and epicardium into the American Heart Association (AHA) 17 segments (58), with 3 features per segment characterizing perfusion and wall thickness. However, the trained boosting classifier showed only moderate accuracy for patientwise prediction of abnormal FFR values. For the same purpose, Zreik et al. (53) trained an SVM based on features from myocardial regions extracted from CCTA. Clinical evaluation of this method yielded improved diagnostic accuracy of FFRCT over visual evaluation of stenosis (79). Hae et al. (51) increased accuracy of FFR-prediction by including the tissue volume subtended to a stenotic lesion in analysis. However, determination of lesion position required additional analysis including artery tree segmentation.

### 5. DISCUSSION

We have presented a survey of applications of ML for detection, characterization and quantification of atherosclerotic plaque and stenosis in cardiac CT. We found that while ML has been a mainstay of cardiac image analysis for years, the recent emergence of deep learning has accelerated progress in the field. Machine learning has the potential to unburden clinicians from time-consuming tasks and change diagnostic procedures, thereby reducing healthcare costs. Moreover, low-cost ML-based analysis could be added to screening studies as a secondary goal. In this survey, we have focused on ML for CAD analysis. For a broader scope the reader is referred to Al'Aref et al. (80), Litjens et al. (81), Nicol et al. (82), Petersen et al. (83), and Singh et al. (84).

We have reviewed plaque and stenosis analysis methods in separate sections, but formation of plaque and stenosis is naturally related and many papers have proposed simultaneous analysis [e.g., (48, 53)]. Moreover, (semi-)automatic identification of plaque or stenosis is often only an intermediate step for prediction of cardiovascular events. Motwani et al. (85) used stenosis scores and plaque characteristics to develop a model for 5 years all-cause mortality prediction. Similarly, Johnson et al. (86) showed that an ML model taking into account per segment coronary artery characteristics can outperform hand-crafted models for prediction of adverse cardiac events. Van Rosendael et al. (87) developed a model for all-cause mortality prediction in combination with future myocardial infarction based only on hand-crafted features derived from CCTA scans. Furthermore, some methods directly predict presence of CAD from medical images, i.e., chest CT (88) or non-contrast-enhanced cardiac CT (89). While these approaches only require one label per patient and large data sets are thus not expensive to obtain, the interpretability of predictions may be limited. Interpretability might constitute an opportunity, not only to improve reliability but also as it might increase

### REFERENCES

medical knowledge by quantifying the diagnostic relevance of underlying phenomena.

The readiness of automatic analysis methods for clinical implementation depends on the complexity of the task, but also on other factors. ML algorithms require large training sets, and tasks with abundant data may be easier to automate. For example, obtaining a ground truth for e.g., non-calcified plaque characterization is very challenging. Therefore, data sets are generally small and ML algorithms remain at an early developmental stage. In contrast, large data sets are available for the development of ML-based CAC scoring methods, which has led to highly accurate results in both dedicated cardiac CT images (11) and other CT images visualizing the heart (8, 32). Similarly, ML-based FFRCT development is aided by the availability of large data sets with CFD-derived reference values. An important remaining step toward clinical application of FFRCT lies in performance evaluation specifically for subjects around the FFR threshold of 0.8, which were shown to be most challenging (90). Furthermore, a recent study showed that not all CCTA exams are suitable for FFRCT analysis (78).

Many challenges in the adoption of machine learning methods in the clinic are not exclusive to CAD detection in cardiac CT. For example, ML algorithms could show unexpected behavior, motivating research into ML interpretability and explainability (91). Furthermore, it is important to point out that ML algorithms are often trained and evaluated on single center studies with high risk for selective biases, and under exclusion of low quality scans.

Despite these challenges, current rapid development allows for justifiable hope that the importance of ML algorithms in cardiac CT will not cease to increase in near future, with benefits for clinicians and patients alike.

### AUTHOR CONTRIBUTIONS

NH and SV drafted the manuscript, which was critically revised and edited by JW, TL, and II. Authors agree to be accountable for all aspects of the work.

### FUNDING

This study has been financially supported by Pie Medical Imaging BV.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcvm. 2019.00172/full#supplementary-material

as the first-line test for stable chest pain. Heart. (2018) 104:921–7. doi: 10.1136/heartjnl-2017-311846

**Conflict of Interest:** II and TL received institutional research projects by Dutch Technology Foundation cofunded by Pie Medical Imaging and Philips Healthcare (P15-26), the Netherlands Organisation for Health Research and Development with participation of Pie Medical Imaging (104003009). II received institutional research project by Dutch Technology Foundation cofunded by Pie Medical Imaging (12726). II and TL are cofounders and shareholders of Quantib-U BV.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Hampe, Wolterink, van Velzen, Leiner and Išgum. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Machine Learning Approaches for Myocardial Motion and Deformation Analysis

Nicolas Duchateau1 \*, Andrew P. King2 and Mathieu De Craene3

1 CREATIS, CNRS UMR 5220, INSERM U1206, Université, Lyon, France, 2 School of Biomedical Engineering and Imaging Sciences, King's College London, London, United Kingdom, 3 Philips Research Paris, Suresnes, France

Information about myocardial motion and deformation is key to differentiate normal and abnormal conditions. With the advent of approaches relying on data rather than pre-conceived models, machine learning could either improve the robustness of motion quantification or reveal patterns of motion and deformation (rather than single parameters) that differentiate pathologies. We review machine learning strategies for extracting motion-related descriptors and analyzing such features among populations, keeping in mind constraints specific to the cardiac application.

#### Edited by:

Steffen Erhard Petersen, Queen Mary University of London, United Kingdom

#### Reviewed by:

Antonio De Marvao, Imperial College London, United Kingdom Wenjia Bai, Imperial College London, United Kingdom Alireza Sojoudi, University of Calgary, Canada

\*Correspondence: Nicolas Duchateau<EMAIL_ADDRESS>
#### Specialty section:

This article was submitted to Cardiovascular Imaging, a section of the journal Frontiers in Cardiovascular Medicine

> Received: 14 October 2019 Accepted: 12 December 2019 Published: 09 January 2020

#### Citation:

Duchateau N, King AP and De Craene M (2020) Machine Learning Approaches for Myocardial Motion and Deformation Analysis. Front. Cardiovasc. Med. 6:190. doi: 10.3389/fcvm.2019.00190 Keywords: machine learning, computer-aided diagnosis, myocardial motion, myocardial strain, cardiac imaging

## 1. INTRODUCTION

### 1.1. Myocardial Motion and Deformation Analysis: What For?

Pump efficiency can discriminate failing from healthy hearts, as quantified by volume and ejection fraction. Clinicians are well aware of the limitations of these simple measurements to face the complexity of heart disease, and recommend finer markers of cardiac mechanical dysfunction (1). Myocardial motion (displacement or velocity) and deformation (strain or strain rate) are richer descriptors of (ab)normal cardiac function (2, 3). They can provide characteristic spatiotemporal signatures for disease at each location of the myocardium and each instant of the cardiac cycle. They are often projected onto anatomically-relevant directions to facilitate interpretations (4). Interestingly, they can be estimated from routine modalities such as echocardiography and magnetic resonance (MR) (5), and have therefore been thoroughly investigated for a wide range of applications.

### 1.2. Machine Learning for Myocardial Motion and Deformation Analysis: What For?

Machine learning builds upon models whose optimal parameters are learnt from a set of samples representative of the studied population. This data-driven approach is more flexible than traditional methods (e.g., variational), as demonstrated for myocardial segmentation (6, 7), and has strong potential for the analysis of complex descriptors such as myocardial motion and deformation. In essence, machine learning seeks to learn data representations (either explicit or hidden) for better solving a supervised problem or for characterizing the data distribution. This often involves dimensionality reduction to facilitate the analysis of high-dimensional descriptors, and requires navigating between the low-dimensional/latent space and high-dimensional/original space for better interpretation.

### 1.3. Which Data Approach for Learning?

Over the years, researchers have gained detailed knowledge of the complexity of cardiac mechanics, and proposed physiologicallyrelevant motion and deformation descriptors, from global strain in a single anatomical direction to richer representations such as 3D+t vector or tensor fields. Most approaches decompose the analysis into two steps (**Figure 1A**): the extraction of motion/deformation descriptors from image sequences, followed by their analysis over a population of interest. Machine learning can address both parts, and we discuss these topics separately (sections 2 and 3). Deep neural networks (8) may address the two parts in **Figure 1A**, but also enable the analysis of population data directly from the image sequences by looking for image features not necessarily interpretable or visualizable, but optimal to answer the clinical question of interest (**Figure 1B**). We specifically comment on this strategy, which is more recent and preliminary, in section 4.7.

## 2. MOTION AND DEFORMATION ESTIMATION

Traditionally, myocardial motion fields have been estimated from images using standard image registration techniques such as optical flow (9), free-form deformation (10), or block matching (11). Naturally, this depends on the algorithm ability to catch motion-related structures, which strongly varies with the imaging modality. Tags and speckles can directly be tracked within the myocardium in tagged MR and 2D/3D echography (within the limits of tag fading, speckles temporal consistency, and out-of-plane motion), contrary to cine MR where algorithms tend to approximate motion from endocardial/epicardial contour tracking. A dedicated review (5) details the standards for spatial and temporal resolution and the influence of imaging parameters on the estimation of myocardial deformation.

Approaches based on neural networks challenge the variational formulation of motion estimation, as shown on video image sequences with the FlowNet2 convolutional neural network (CNN) architecture (12) that focuses on optical flow. Similar approaches have been applied to cardiac imaging (13, 14), but raise several methodological questions. First, the generalization ability of the trained networks to estimate a wide range of deformations at multiple scales still needs to be verified. This is critical for specific disease traits of lower prevalence. Furthermore, robustness to a variety of routine clinical imaging conditions (different image qualities, fields of view, devices, etc.) needs to be established. Second, supervised CNN-based motion estimators such as FlowNet2 do not embed any regularization, and are therefore sensitive to imaging noise if it differs from the training database. This not the case for unsupervised approaches like (13), which use an intensity-based loss, combined with a regularization term as in classical image registration. Finally, motion features can boost segmentation performances (15–17), as looking at several frames improves the manual segmentation of physicians. Further details are given in a review dedicated to deep learning for motion estimation in medical imaging (18).

Statistical models learnt from data can act as regularizers for tracking algorithms. (19) used dictionary learning as a sparse basis for cardiac motion fields to feed the regularization. Within deep learning, auto-encoders can encode spatial transformations into a low-dimensional space and provide powerful projection and reconstruction operators to connect with the tracking in the original image space (20).

Additional constraints specific to the cardiac application can provide more plausible registration outputs, such as invertibility (the myocardium does not fold) and incompressibility, as investigated for the diffeomorphic LogDemons (21) and freeform-deformation algorithms (22). Temporal consistency has been enforced through 4D representations of motion (23, 24), for multiple pairwise transformations simultaneously (25), or for intra/inter-subject mappings (26). Motion and deformation estimation with machine learning should also consider these aspects for better consistency and robustness.

## 3. MOTION AND DEFORMATION ANALYSIS

### 3.1. Before the Analysis: Data Normalization

Cardiac image data often need to be normalized in terms of anatomy, frame rate or cycle phases, before any statistical or machine learning analysis.

Image sequences can be registered using a 4D transformation model based on e.g., free-form deformation (10) or demons (26). This approach quantifies the spatiotemporal differences between the image sequences, analyzed statistically afterwards through deformation-based morphometry methods.

Motion or deformation descriptors (or any other data) from a given individual can also be transported to a reference template (generally, a central case at end-diastole). This involves local reorientation of the motion/deformation fields (27, 28), adjusted to the addressed clinical question (29). Temporal differences between sequences can also be normalized by resampling before the motion extraction [e.g., piece-wise linear interpolation (30)]. Recent approaches transport the whole subject-specific trajectory instead of the descriptors of interest, with specific computational considerations (31, 32). Automatically estimating multiple templates across the sequence may also be well adapted to the cardiac circular/periodic dynamics (33).

In both strategies, existing data correspondences facilitate the normalization. Spatial alignment can rely on anatomical landmarks (apex, valve ring, etc.) or point-to-point correspondences obtained from model-based tracking of the anatomy. Temporal alignment can use physiologically-relevant instants, such as the maximum contraction (10) or QRS and valve events (28).

### 3.2. Learning From Motion and Deformation Data

Machine learning can benefit a wide range of clinical problems. Unsupervised approaches learn a data representation that uncovers useful insights into the data distribution, but without explicit reference to a particular clinical question. Clustering and dimensionality reduction techniques fall into this category. Supervised approaches train a model for a specific task, and labels/annotations are provided as supervision. For example, diagnosing disease may involve binary labels for supervision (disease/healthy) and the task would be to predict these labels from the motion data. The type of labels determines the task addressed by the model: categorical labels mean classification, whereas discrete or continuous labels imply regression. Supervised approaches also involve learning a (lower dimension) representation of the data that facilitates the classification/regression, but this representation can be formed in an unsupervised or supervised way, as described below.

### 3.2.1. Unsupervised Learning

Unsupervised motion and deformation analysis shares objectives with statistical atlases, regarding how to characterize variability across a population. Pioneering works directly applied a principal component analysis (PCA) on myocardial displacements at each spatiotemporal location (34) over a healthy population, later extended through the estimation of local abnormalities in the myocardial velocities of a given subject compared to a reference population (28, 35). However, these analyses consider each spatial location and temporal instant independently from the others. The statistical analysis can also consider the motion patterns over the entire cardiac cycle as high-dimensional objects, as simply demonstrated through a PCA on temporal strain traces concatenated over the heart segments (36, 37). This approach reminds earlier work on Active Appearance Motion Models (38), which statistically analyzed both displacement and image intensity information over the entire cardiac cycle.

More advanced strategies estimate a low-dimensional space that encodes the high-dimensional myocardial motion/deformation data and navigate through this space, although this requires specific care. Myocardial shapes across a population can be considered as originating from one or several references under the action of a transformation such as a diffeomorphic warping. In this case, the space of myocardial shapes is related to the (known) non-linear high-dimensional space of diffeomorphic transformations. This space is a manifold, and known tools exist to perform statistics on such transformations and therefore on myocardial shapes while preserving this data structure (39, 40). Myocardial motion/deformation patterns may also be considered as originating from a non-linear high-dimensional manifold, but in this case the manifold is unknown. Machine learning allows estimating this space from data, and can overcome the limitations of linear techniques such as PCA that ignore this known structure. A general framework (41) groups the vast variety of existing manifold learning techniques. A graph is built across high-dimensional samples to approximate the manifold, and diagonalization, and dimensionality reduction processes provide a low-dimensional space that encodes the data. Techniques generally differ on how input samples are related within the graph, either locally (e.g., distance between neighbors, or local structure variations expressed in the graph Laplacian) or globally (e.g., geodesic distance). These techniques improve the statistical analysis of myocardial motion and deformation patterns. They can represent the continuum of disease from normality while preserving the data structure (42). The unsupervised representation of populations is particularly interesting when existing labels are not fully trusted, as in heart failure with preserved ejection fraction (43, 44) or when a supervised formulation of the clinical problem is uncertain, such as outcome from cardiac resynchronization therapy (45).

Nonetheless, these techniques normally lack explicit mappings between the high-dimensional and low-dimensional spaces, which are typically approximated using out-of-sample reconstruction/regression (46) and are therefore inexact. Deep learning auto-encoders explicitly address this by simultaneously learning how to encode and decode high-dimensional data with a limited number of parameters while minimizing the reconstruction error. However, this also requires constraining the distribution of samples in the latent space so that a statistical analysis can still be performed on it afterwards, as in variational auto-encoders (47). These techniques are promising for the analysis of myocardial motion and deformation and start being used in cardiac imaging for segmentation (48, 49) or segmentation-based biomarkers (50).

#### 3.2.2. Supervised Learning

As noted above, designing a supervised learning model traditionally consists of two steps (**Figure 1A**). First, the input data are transformed to a new representation that facilitates the task performance. Second, a classification or regression model is trained to predict the label given the new representation. More recent techniques such as deep learning combine these two steps: the representation is learnt and optimized during the model training (**Figure 1B**). Below, we first summarize works using supervised learning in the traditional way and then we briefly review more recent deep learning approaches.

The new data representation can be estimated using knowledge of the labels (supervised way) or without such knowledge (unsupervised). In other words, although the final classification or regression model is supervised, the transformation to a new representation can be unsupervised. Examples include the dimensionality reduction methods reviewed in section 3.2.1, such as PCA (51–53) or non-linear manifold learning (53, 54). The use of hand-crafted features such as volumes/diameters/strains (55) and radius/thickness (56, 57) also falls into this category, although one could argue that knowledge of the task was also used to design these features. A supervised approach was taken in Dawes et al. (58), in which supervised PCA was used to find the principal components of displacement data related to survival.

Classification or regression come once the new representation is obtained. Many classification algorithms have been used, including support vector machines (SVM) (55, 59), random forests (55), variants of dictionary learning (59–61) and ridge logistic regression (57). Regression applications rely on svm (62) and multiscale kernel regression (54).

Recent research has increasingly focused on deep learning for both classification and regression from dynamic imaging data. In these approaches, the activations of intermediate network layers can stand as a transformed representation formed in a supervised way. Inputs to these models are commonly dynamic image intensity data, but segmentation data has also been used (63). For classification, variants of auto-encoders have been a common architecture choice. An auto-encoder is a deep learningbased dimensionality reduction technique, and classification can be performed in the low-dimensional latent space learnt without supervision (53), or in a supervised way by including classification accuracy into the loss function (48, 63, 64). Autoencoders are attractive as they allow examining the classification features in the original image space, leading to more interpretable analyses. CNNs have also been proposed for classification (65), and a challenge on automated diagnosis was recently organized (7). Regression tasks such as estimating volume and/or ejection fraction may also involve CNNs (66), as tested on the recent Kaggle Challenge data1 . Variational auto-encoders have also been used to perform regression in the latent space (50).

A wide set of classification applications involved myocardial motion or deformation, including identifying abnormal wall motion (59, 61), predicting therapy response (67) and survival (58, 64), and diagnosing myocardial infarct (16, 60, 65, 68) or pathology (7, 48, 57, 63). Regression applications aimed at localizing myocardial infarct (54), grading myocardial motion defects (62), and estimating volumes (66).

Detecting some form of abnormality is a common theme for supervised learning applications, for which two main strategies exist. In the first one, the transformed representation only involves healthy subjects: the distribution of samples in the low-dimensional space therefore represents healthy variations, and subsequent subjects who fall away from the healthy distribution are considered abnormal, as investigated on myocardial velocities (28, 35) and shapes (69). The other strategy learns a low-dimensional representation from both healthy and pathological subjects, where supervised classification can be applied afterwards (70).

### 4. SPECIFICITIES OF THE CLINICAL CONTEXT

### 4.1. Physiological Consistency

Learning algorithms utilize a low-dimensional representation of the high-dimensional motion/deformation data, where the population variability is either rendered through diagonalization according to inter-subject distances, or correlated to labels of interest. Transforming to and from this representation involves interpolation between samples. Regularizing the lowdimensional space ensures smoother interpolation and generates new samples that are physiologically plausible (49, 71). In both of these works, the low-dimensional space produced by the encoding part of a CNN was regularized to map smoothly to a set of input shapes, labeled images, or slice locations. This notion of joint projection from the image and label space is also inherently present in more classical manifold learning techniques such as partial least squares. Similar notions need to be extended to motion fields, whilst mapping similar pathological conditions to close locations in the latent space.

### 4.2. Spatiotemporal Analysis

Most learning techniques consider high-dimensional inputs as high-dimensional column vectors or a set of patches, and disregard the spatiotemporal characteristics of motion and deformation. Few works explicitly addressed this issue for the statistical analysis of populations. A bilinear statistical model was used on cardiac shapes (72) to distinguish intersubject variations from individual heart dynamics. (73, 74) explicitly addressed the problem through spatiotemporal tensor decomposition. Duchateau et al. (75) tuned up the contributions of the spatial, temporal, and magnitude dimensions to analyze changes in deformation patterns through registration. Jia et al. (31) and Guigui et al. (32) transported temporal trajectories without explicitly extracting motion or deformation descriptors beforehand. These strategies, limited to variability analyses, pave the ground for better considering spatiotemporal aspects with machine learning.

1Available online at: https://www.kaggle.com/c/second-annual-data-sciencebowl/data

### 4.3. Interpretability

Many tasks may benefit from somehow "interpretable" learnt models, i.e., a user should have ways to inspect the input data characteristics that led to the output prediction or representation. The recent trend toward more complex learning models (such as deep learning) has raised the interest for this property, since these models are generally harder to interpret than simpler ones. One approach consists in defining a simpler model that is "locally similar" to the global complex model (i.e., it has similar performance for similar inputs) (76). For deep learning based approaches, "saliency maps" can be produced, which show which parts of the input data were important in producing the output. Alternatively, regression or autoencoders can be used to reconstruct cases from the low-dimensional latent space and examine features in the original-high dimensional space, with clear benefits for interpretability as demonstrated in Clough et al. (48), Puyol-Anton et al. (53), Biffi et al. (63), and Bello et al. (64).

machine learning for myocardial motion or deformation analysis.

### 4.4. Database Size and Heterogeneity

Traditionally, difficulties in accessing and reliably annotating databases of medical images have led to smaller databases in medical imaging compared to computer vision applications. Recent initiatives such as the UK Biobank project2 (77) now provide large-scale annotated imaging databases, fuelling a rise in more data-intensive methods such as deep learning. **Figure 2** illustrates this high increase over recent years for the studies reported in this paper. The impact of these large databases is high: reporting reference ranges for cardiac functional biomarkers is now possible with much greater confidence (78, 79), in addition to detecting effects otherwise hidden with smaller databases, as shown for genome data (77). Data heterogeneity is also crucial when choosing or curating a database for a specific task, i.e., the database should include sufficient subjects to cover a

2Available online at: https://www.ukbiobank.ac.uk/

range of values for the output label and guarantee the model generalizability. More pathology-focused databases such as those in the Cardiac Atlas Project3 (80) have an important role to play in this respect.

### 4.5. Validation and Standardization Initiatives

As analyzing the tracking output is sensitive to processing errors, in particular for multi-centric data, tracking algorithms should be benchmarked to prevent bias due to different manufacturers or settings/practices. To ensure reproducibility of clinical decisionmaking from these data, standardization initiatives arose from academic, clinical, and industrial actors of cardiac imaging. Strain estimation was compared across vendors for synthetic and real images (81). Outputs were consistent regarding the differentiation between pathological and healthy regions, and the identification of ambiguous zones. However, statistically significant differences among vendors were reported, including differences around 15% for the biggest scars. These differences call for benchmarks on more realistic datasets (both regarding geometry and image quality), obtained e.g., from simulation frameworks that mix image formation and biomechanical models with real images (82).

Complementary standardization of imaging are also investigated through deep learning, for the control of e.g., the full coverage of the ventricles (83), the view/plane (84, 85), and the image quality in general (78) or due to motion-related artifacts (86).

### 4.6. Multiple Modalities/Descriptors

Most studies only consider a single type of motion or deformation descriptor at once from a single acquisition and a single modality, unlike clinical reasoning, which repeats acquisitions in the same or different modality and uses different types of measurements and descriptors. Recent works addressed these limitations within the framework of manifold learning. (30) enforced the complementarity of multimodal acquisitions (tagged MR and 3D echocardiography) using canonical correlation analysis and partial least squares methods. (87) used a similar strategy to better relate myocardial shape and deformation descriptors. Puyol-Anton et al. (70) investigated multi-view linear discriminant analysis for classification purposes. Finally, the more generic framework of multiple kernel learning allows reducing the dimensionality and examining the weights attributed to each descriptor. It was applied to supervised (67) and unsupervised (43–45, 88) problems, to investigate multiple descriptors among which motion-based ones, which could come from different modalities or different views of a single modality.

### 4.7. Complexity of the Models and Data Descriptors

Machine learning relies on models whose complexities should be adjusted to the question being answered. Researchers should keep in mind that such models only provide an approximation of reality, and try to minimize this error (e.g., by refining the model, adding more data or relevant descriptors, or estimating uncertainties). We strongly recommend to start with simple data descriptors and models, and carefully benchmark the retained methods against simpler models or even standard statistics.

Deep learning approaches allow circumventing the design of hand-crafted features (**Figure 1B**), and therefore go beyond a substantial limitation of standard machine learning. They mainly have been used for supervised problems and avoiding segmentation. The ACDC challenge (7) included a diagnosis challenge not necessarily requiring segmentation, although all participants opted for segmentation-based diagnosis. Regressionbased estimation of cardiac parameters directly from images was proposed in (66, 89, 90), and may also strengthen the segmentation-based estimation of such parameters (91). However, as already pointed out, this direct strategy may also limit interpretability, and therefore transfer to clinical practice.

## 5. CONCLUSION

Machine learning offers wide possibilities to automate processing, and notably extract and analyze myocardial motion and deformation. Driven by advances in cardiac segmentation and large databases collection, there is potential for substantially improving the characterization of the cardiac function and impacting clinical practice. Changes cover the automation of time-consuming and user-dependent tasks such as feature extraction, higher performance on supervised problems such as (earlier) diagnosis, prognosis, and risk stratification, and new unsupervised data representations for knowledge discovery such as clustering or phenotyping. Nonetheless, motion and deformation are rich but complex high-dimensional data. Efforts need to be continued to reduce uncertainties, approximations, and crucial misinterpretations along the analysis pipeline, from careful problem definition, compliance with the mathematical and physiological data properties, algorithms benchmarking/validation/testing, and health actors' education.

### AUTHOR CONTRIBUTIONS

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

### FUNDING

ND was partially supported by the LABEX PRIMES of Université de Lyon (ANR-11-LABX-0063) and the MIC-MAC JCJC project (ANR-19-CE45-0005). AK was partially supported by the Wellcome EPSRC Centre for Medical Engineering at Kings College London (WT 203148/Z/16/Z) and the UK EPSRC (EP/P001009/1 and EP/R005516/1). MD was partially supported by the European Union Horizon 2020 Programme for Research and Innovation (CardioFunXion, H2020-14-MSCA-ITN-642676).

3Available online at: https://www.cardiacatlas.org/

### ACKNOWLEDGMENTS

The articles discussed in this review were selected by querying PubMed over the last 10 years with the terms (myocardial [OR] cardiac) [AND] learning [AND] (motion [OR] deformation), complemented by the authors' knowledge, and examining the

### REFERENCES

publication profile of the authors of the already selected articles. Papers not using spatial or temporal motion or deformation patterns but single measurements such as peak values or timings, and papers addressing cardiac respiratory motion were removed from this selection, although we acknowledge their importance for the more complete analysis of cardiac function.

Workshop, MIL3ID 2019, Shenzhen, Held in Conjunction with MICCAI 2019. Shenzhen (2018) 11071:472–80. doi: 10.1007/978-3-030-00934-2-53

from MR and ultrasound data. Med Image Anal. (2017) 40:96–110. doi: 10.1016/j.media.2017.06.002

**Conflict of Interest:** MD was employed by Philips Research Paris. ND and AK have research publications and/or projects with researchers and engineers from private companies, but this did not influence the contents of this review.

Copyright © 2020 Duchateau, King and De Craene. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Artificial Intelligence for Cardiac Imaging-Genetics Research

#### Antonio de Marvao† , Timothy J. W. Dawes † and Declan P. O'Regan\*

*MRC London Institute of Medical Sciences, Imperial College London, London, United Kingdom*

Cardiovascular conditions remain the leading cause of mortality and morbidity worldwide, with genotype being a significant influence on disease risk. Cardiac imaging-genetics aims to identify and characterize the genetic variants that influence functional, physiological, and anatomical phenotypes derived from cardiovascular imaging. High-throughput DNA sequencing and genotyping have greatly accelerated genetic discovery, making variant interpretation one of the key challenges in contemporary clinical genetics. Heterogeneous, low-fidelity phenotyping and difficulties integrating and then analyzing large-scale genetic, imaging and clinical datasets using traditional statistical approaches have impeded process. Artificial intelligence (AI) methods, such as deep learning, are particularly suited to tackle the challenges of scalability and high dimensionality of data and show promise in the field of cardiac imaging-genetics. Here we review the current state of AI as applied to imaging-genetics research and discuss outstanding methodological challenges, as the field moves from pilot studies to mainstream applications, from one dimensional global descriptors to high-resolution models of whole-organ shape and function, from univariate to multivariate analysis and from candidate gene to genome-wide approaches. Finally, we consider the future directions and prospects of AI imaging-genetics for ultimately helping understand the genetic and environmental underpinnings of cardiovascular health and disease.

Keywords: artificial intelligence, machine learning, deep learning, genetics, genomics, imaging-genetics, cardiovascular imaging, cardiology

### INTRODUCTION

Cardiovascular conditions remain the leading cause of mortality and morbidity worldwide (1), with genetic factors playing a significant role in conferring risk for disease (2). High-throughput DNA sequencing and genotyping technologies, such as whole-genome sequencing and high-resolution array genotyping, have developed at an extraordinary pace since the first draft of the human genome was published in 2001 at a cost of \$0.5-1 billion (3). Continuous improvements have so far outpaced Moore's law, with the sequencing cost per genome currently estimated to be \$1,000 (4), enabling cost-effective sequencing of millions of humans. At the same time, technological advances in physics, engineering, and computing have enabled a step-change improvement in cardiovascular imaging, facilitating the shift from one dimensional, low-fidelity descriptors of the cardiovascular system to high-resolution multi-parametric phenotyping. These capabilities are not limited to research settings but are increasingly available in clinical echocardiography, nuclear imaging, computerized tomography (CT), and cardiovascular magnetic resonance (CMR) practice. An unprecedented volume of clinical data is also becoming available, from smartphone-linked wearable sensors (5) to the numerous variables included in the electronic health records of entire

#### Edited by:

*Steffen Erhard Petersen, Queen Mary University of London, United Kingdom*

#### Reviewed by:

*Alexander Teumer, University of Greifswald, Germany Avan Suinesiaputra, The University of Auckland, New Zealand*

#### \*Correspondence:

*Declan P. O'Regan<EMAIL_ADDRESS>
*†These authors have contributed equally to this work and share first authorship*

#### Specialty section:

*This article was submitted to Cardiovascular Imaging, a section of the journal Frontiers in Cardiovascular Medicine*

Received: *30 September 2019* Accepted: *27 December 2019* Published: *21 January 2020*

#### Citation:

*de Marvao A, Dawes TJW and O'Regan DP (2020) Artificial Intelligence for Cardiac Imaging-Genetics Research. Front. Cardiovasc. Med. 6:195. doi: 10.3389/fcvm.2019.00195* populations (6). However, the volume, heterogeneity, complexity, and speed of accumulation of these datasets now make humandriven analysis impractical. Artificial intelligence (AI) methods such as machine learning (ML), are particularly suited to tackling the challenges of "Big Data" and have shown great promise in addressing complex classification, clustering, and predictive modeling tasks in cardiovascular research. Cardiac imaginggenetics refers to the integrated research methods that aim to identify and characterize the genetic variants that influence functional, physiological, and anatomical phenotypes derived from cardiovascular imaging.

In the same way that basic statistical literacy has become a routine aspect of clinical practice, a basic understanding of AI's strengths, applications, and limitations is becoming essential for practicing researchers and clinicians. Here we introduce common AI principles, review applications in imaging-genetics research, and discuss future directions and prospects in this field.

### IMAGING-GENETICS: FROM SINGLE GENE HYPOTHESIS-TESTING TO GENOME-WIDE HYPOTHESES GENERATION

Imaging-genetics aims to dissect and characterize the complex interplay between imaging-derived phenotypes and environmental and genetic factors. Many principles and approaches originated from neuroimaging research, where the first attempts at integrating multi-parametric phenotypes, obtained from structural and functional brain MRI, with genetic data were carried out (7). To help manage the computational and statistical challenges inherent to the use of "Big Data" squared (high-dimensional imaging × high-dimensional genetic data), interrogations were limited to pre-defined regions of interest in the brain and candidate genes or SNPs, based on a priori assumptions about the biology of disease (8). Similar, "hypothesis-led" designs underpinned candidate gene and linkage studies that established causal relationships between rare genetic variants and rare conditions, such as those that first identified the role of myosin heavy-chain beta in hypertrophic cardiomyopathy (HCM) (9) and of titin in dilated cardiomyopathy (DCM) (10).

The increased affordability of DNA sequencing and genotyping resulted in genetic information becoming available in large numbers of subjects. This has contributed to shift the focus to genetic discovery and the study of common, complex disease traits. These traits are not characterized by a single gene mutation leading to a large change on the phenotype but attributable to the cumulative effects of many loci. Although the effect sizes of individual loci are relatively modest, composite effects can significantly alter the probability of developing disease (11). The "common disease—common variant" hypothesis underpins genome wide association studies (GWAS), where subjects are genotyped for hundreds of thousands of common variants. For example, a study into the genetic determinants of hypertension in over 1 million subjects, identified 901 loci that were associated with systolic blood pressure (SBP) and these explained 5.7% of the variance observed (12). Even though these single nucleotide polymorphisms (SNPs) explain only a small proportion of phenotypic variance they provide relevant, hypothesis-generating biological or therapeutic insights. The rapid development of complementary high-throughput technologies, able to characterize the transcriptome, epigenome, proteome, and metabolome now enables us to search for molecular evidence of gene causality and to understand the mechanisms and pathways involved in health and disease (13). These large biological multi-omics data sets and their computational analysis are conceptually similar to the more established study of genomics and examples of such work are included in this review.

### IMAGING-GENETICS: FROM ONE-DIMENSIONAL PHENOTYPING TO MULTIPARAMETRIC IMAGING

Several biological and technical reasons have been proposed to explain the "missing heritability" of complex cardiovascular traits. However, a common factor limiting many genotypephenotype studies was that the ability to characterize phenotypes rapidly and accurately, significantly lagged behind our ability to describe the human genotype (14). Phenotyping was characterized by imprecise quantification, sparsity of measurements, high intra- and inter- observer variability, low signal to noise ratios, reliance on geometric assumptions, and adequate body habitus, poor standardization of measurement techniques and the tendency to discretize continuous phenotypes (15). Commonly, the complexity of the cardiovascular system was distilled into a small number of continuous one-dimensional variables [e.g. volumetric assessment of the left ventricle (16)] or, convenient dichotomies, such as responders vs. non-responders (17), leading to a loss of statistical power (18).

The imaging community responded to calls for more accurate and precise, high-dimensional phenotyping (19, 20) with the roll out of developments in echocardiography (e.g., tissue doppler, speckle-tracking, and 3D imaging), CMR (e.g., tissue characterization, 4D flow, 3D imaging, diffusion tensor imaging, spectroscopy, and real-time scanning), CT (e.g., improved spatial and temporal resolution, radiation dose reduction techniques, functional assessment of coronary artery flow using FFR-CT, and coronary plaque characterization), and nuclear cardiology (e.g., improvements in radiopharmaceuticals and hardware resulting in increased accuracy and reduced radiation exposure). In parallel, computational approaches have become increasingly integral to the clinical interpretation of these much larger datasets (21–23) and several have obtained FDA approval (24).

### IMAGING-GENETICS: A "BIG DATA" SQUARED PROBLEM

Leveraging these deeper phenotypes is an attractive proposition but the joint analysis of high-dimensional imaging and genetic data poses major computational and theoretical challenges. An early example of a neuroimaging GWAS investigated the association between 448,293 SNPs and 31,622 CMR voxels in a cohort of 740 subjects (25). This study highlighted difficulties correcting for multiple testing (1.4 × 1010 tests were performed) and the need for unprecedented computational power (300 parallel cores).

Simultaneously assessing the statistical significance of several hundred thousand tests vastly increases the number of anticipated type I errors. If the probability of incorrectly rejecting the null hypothesis in one test with a pre-set α of 0.05 is 5%, then under the same conditions, the probability of incorrectly rejecting the null hypothesis at least once if 100 tests are performed is 99.4%. Therefore, an adjustment for the number of tests being carried out is required. The simplest approach for adjustment for multiple testing is the Bonferroni correction, where the pre-set α is recalculated as α/m, where m represents the number of independent tests being performed. However, this method is overly conservative when m is large, leading instead to many false negatives. An alternative, extensivelyvalidated method is the Benjamini–Hochberg Procedure (26). Using this approach, instead of controlling for the chance of any false positives, an acceptable maximum fixed percentage of false discoveries (the expected proportion of rejected hypotheses that are false positives) is set.

A further consideration in the statistical analysis of highdimensional cardiac phenotypes is that a clinically significant signal will not originate from a single voxel but across many voxels in extended, anatomically coherent areas. Indeed, approaches such as threshold-free cluster enhancement (TFCE), which were developed in neuroimaging (27), have recently applied in cardiovascular research (28). Using such methods, both signal size and contiguity with surrounding signal patterns contribute to inference statistics.

### ARTIFICIAL INTELLIGENCE

Artificial intelligence, machine learning, and deep learning are terms that are interlinked, have some overlap but are often incorrectly used interchangeably. AI refers to the overarching field of computer science focused on simulating human cognitive processes. As a subset of AI, machine learning refers to the family of algorithms that share a capacity to perform tasks like classification, regression, or clustering based on patterns or rules iteratively learnt directly from the data without using explicit instructions. ML algorithms can be further subdivided into supervised, unsupervised, and reinforcement learning.

Supervised learning is the most common form of traditional ML and involves the training of models on pairs of input and expected outputs ("labeled" data) and then their deployment to make predictions in previously unseen data. It includes such approaches as nearest neighbor, support vector machines, random forests and naïve Bayes classifiers. Unsupervised learning algorithms are used to address clustering or dimensionality reduction problems by detecting patterns and structures within the data without any prior knowledge or constraints. In other words, the model organizes "unlabeled" data into groupings that share common, previously undefined characteristics. Examples including k-means clustering, tdistributed stochastic neighbor embedding (t-SNE), and association rule learning algorithms. The use of reinforcement learning algorithms (e.g., deep Q networks), common in robotics and gaming applications (29) has now also been trialed in the navigation of 3D datasets for anatomical landmark detection (30).

Deep learning (DL) is a specific ML method inspired by the way that the human brain processes data and draws conclusions. To achieve this, DL applications use a layered structure of algorithms, called an artificial neural network that imitates the biological neural network of the human brain. The word "deep" in "deep learning" refers to the number of layers through which the data is transformed. The most common DL models are convolutional neural networks (CNN), which are extremely efficient at extracting features and often superior to traditional ML in larger, more complex datasets such as medical imaging and genomics (31, 32). However, feature and process interpretability is more amenable in classical ML as even simple DL networks can operate as "black-boxes." While the computational and time requirements of DL are much higher during training, subsequent inference is extremely fast and DL approaches can be used to accelerate supervised, unsupervised, and reinforcement learning. Indeed, while traditional ML is carried out using central processing units (CPUs), DL was only made possible thanks to the development of graphics processing units (GPUs), which have a massively parallel architecture consisting of thousands of cores and were designed to handle vast numbers of tasks simultaneously.

During the training stage of supervised learning algorithms, the labeled data is divided into training, validation, and testing subsets to reduce overfitting and estimate how well the models generalize. No standard methodologies exist to determine optimum proportions allocated to each set. The training set usually includes a large proportion of the available data and is used for the development of the model. The validation set is used to estimate overall model performance during development and fine-tune the algorithm's hyperparameters (e.g., the number of network layers which could not be learnt). Dividing data into training and validation subsets can be done randomly at the onset of the process or by using a cross-validation approach. This involves dividing the entire dataset into folds of equal size and then training the algorithms in all the folds except one that is left out for validation. The process is repeated until all folds have been used as a validation set and the overall performance of the model is calculated as the average across all validation sets. Finally, an independent (ideally external) test set should be used to assess the model's generalizability.

Despite ML's vast potential and significant performance breakthroughs in fields such as speech recognition, natural language processing, and computer vision, these approaches are not without limitations and vulnerabilities. Some of these are shared with classical statistical approaches (33) while others are entirely novel (34). A significant potential pitfall of ML models derives from the presence of unrecognized confounders that can be present in both the training and testing sets, if they originated from the same dataset. This could result in overfitting of the model to the training data, achieving an artificially inflated performance with poor generalization to other data sets in subsequent studies. The gold-standard approach to address this issue is to obtain a validation dataset acquired by an independent group under real-world conditions. Another possible cause of unsatisfactory generalization of an AI system is if the training data is not an accurate representation of the wider population. For example, an AI model trained on a healthy cohort may not generalize well to a general population that includes extreme disease phenotypes, and a system trained on images from a specific CMR scanner might not perform well when labeling images acquired under different technical conditions. Domain adaptation or transfer learning are fields of AI research that aim to address these challenges.

AI algorithms can also be oversensitive to changes in the input data and therefore vulnerable to unintentional or harmful interference. This was clearly demonstrated in experiments involving "adversarial examples" or inputs that lead the model to make a classification error. For example, the introduction of an imperceptible perturbation in a picture of a benign skin mole resulted in the misclassification as a malignant mole, with 100% confidence (35). The general application of AI has also been hindered by the "black-box" nature of several methodologies. Indeed, full clinical acceptability is only likely if it is possible to explore and scrutinize the predictive features and if the outputs are clinically interpretable.

At a more fundamental level, "Big Data" studies are often no more than observational research. As in classical statistics, observational AI studies cannot test causality and should therefore be considered hypothesis-generating that require further testing. A recent systematic review and meta-analysis of 82 studies applying DL methods to medical imaging found that although the diagnostic performance of DL methods was often reported as equivalent to human experts, few studies tested human vs. DL performance on the same sample and then went on to externally validate their findings (36). Furthermore, apart from a handful of exceptions (37), the effect of AI in routine clinical practice has been rarely tested in the setting of randomized controlled trials. Indeed, it has not been systematically demonstrated that the roll out of AI into clinical practice leads to an improvement in the quality of care, increased efficiency or improved patient outcomes (38). These studies will be required before this technology can be routinely used to help guide clinical care.

**Table 1** provides an introduction to some of the technical and methodological aspects that should be considered in AI research.

Nevertheless, the use of machine learning methods in cardiovascular research has grown exponentially over recent years, with an ever increasing set of uses and applications. Traditional supervised ML methods have been applied successfully to classification tasks in extremely diverse input data, ranging from discrimination between sequences underlying Cis-regulatory elements from random genome sequences (39), separation of human induced pluripotent stem cell-derived cardiomyocytes of distinct genetic cardiac diseases (CPVT, LQT, HCM) (40) to numerous applications in medical imaging analysis. Examples of this include automated

quality control during CMR acquisition (41), high-resolution CMR study of cardiac remodeling in hypertension (42) and aortic stenosis (43), and echocardiographic differentiation of restrictive cardiomyopathy from constrictive pericarditis (44). Unsupervised ML analysis have provided new unbiased insights into cardiovascular pathologies such as by establishing subsets of patients likely to benefit from cardiac resynchronization therapy (45) and by agnostic identification of echocardiography derived patterns in patients with heart failure with preserved ejection fraction and controls (46). Traditional ML has also been used for prediction of outcomes such as hospital readmission due to heart failure (47), survival in pulmonary hypertension (48), and population-based cardiovascular risk prediction (49).

More recently, there has been a greater interest in DL approaches, which have been used with great promise in ever larger-scale classification tasks. Applications include the analysis of CMRs (50), echocardiograms (51), and electrocardiograms (52), identification of the manufacturer of a pacemaker from a chest radiograph (53), aortic pressure waveform analysis during coronary angiography (54); automated categorization of HCM and healthy CMRs (55) and detection of atrial fibrillation using smartwatches (56). DL has also been successfully used to address complex survival prediction tasks in pulmonary hypertension (57) and heart transplantation (58).

The analysis of ever larger and complex genome-scale biological datasets is also particularly suited to ML approaches. One of the strengths of these approaches comes from the ability to discover unknown structures in the data and to derive predictive models without requiring a priori assumptions about, frequently poorly understood, underlying biological mechanisms (59). The field is large, diverse and fast moving with new opportunities for AI to synthesize data and optimize the prediction of key functional biological features appearing all the time. Applications of traditional ML have ranged from the prediction of quantitative (growth) phenotypes from genetic data (60), to the identification of proteomic biomarkers of disease (61), to the prediction of metabolomes from gene expression (62). As in cardiology research, there has been growing interest in applying DL to the field of functional genomics. Such approaches have been used to predict sequence specificities of DNA- and RNA-binding proteins (31, 63), transcriptional enhancers (64) and splicing patterns (65) and to identify the functional effects of non-coding variants (66, 67). A more in depth discussion of the applications of ML and DL to genomics and other multi-omics data can be found elsewhere (68–71).

### ARTIFICIAL INTELLIGENCE IN CARDIOVASCULAR IMAGING-GENETICS

Despite the parallel successes of AI in the fields of genetics and imaging analysis, integrated imaging-genetics research is still an emerging field. However, several studies have already demonstrated the usefulness of AI tools in the analysis of large biological, imaging, and environmental data, in such tasks as dimensionality reduction and feature selection, speech recognition, clustering, image segmentation, natural language processing, variable classification, and outcome prediction (**Figure 1**).

To predict which dilated cardiomyopathy patients responded to immunoglobulin G substitution (IA/IgG) therapy, as assessed by echocardiography, two supervised ML approaches, a random forest analysis and a support vector machine algorithm, were used independently on gene expression data derived from 48 endomyocardial biopsies (72). The overlapping set of 4 genes that was identified by both ML approaches was superior to clinical parameters in discriminating between responders and nonresponders to therapy. The prediction performance was further improved by adding data on the negative inotropic activity (NIA) of antibodies. A support vector machine classifier, also proved to be extremely helpful in identifying specific proteomic signatures that accurately discriminated between patients with heart failure with reduced ejection fraction (HFrEF) and controls in the absence (73) or presence of chronic kidney disease (74). ML pipelines also often use feature selection to more efficiently process high dimensional phenotypes, distinguishing the most informative features from those that are redundant. For example, an information gain method was used to identify speckle-tracking features able to differentiate athlete's heart from HCM. The combination of three different supervised machine learning algorithms (support-vector machine, random forest, and neural network) trained on this sparser data was then shown to be better at distinguishing the two types of remodeling (ML model sensitivity = 87%; specificity = 82%) than conventional echocardiographic parameters (best parameter was e'—sensitivity = 84%; specificity = 74%) (75).

ML approaches have also been successfully used in the identification of new, useful structures in data. One such study, using a hypothesis-free unsupervised clustering approach, revealed four distinct proteomic signatures with differing clinical risk and survival in patients with pulmonary arterial hypertension (76). ML has similarly been able to identify new sub-phenotypes in heart failure with preserved ejection fraction, classifying subjects into three subgroups associated with distinct clinical, biomarker, hemodynamic, and structural groups with markedly different outcomes (77). Okser et al. used a naïve Bayes classifier in a longitudinal imaging-genetics study of 1,027 young adults to identify a predictive relationship between genotypic variation and early signs of atherosclerosis, as assessed by carotid artery intima-media thickness, which could not be explained by conventional cardiovascular risk factors (78).

Classification problems, such as pixel-wise classification of CMR images, are also particularly suited to supervised classical ML (79, 80) and deep learning approaches (81). These highresolution representations of whole-heart shape and function can encode multiple phenotypes, such as wall thickness or strain, at each of thousands of points in the model (82). Such high-fidelity models were used in a study aiming to clarify the physiological role of titin-truncating variants (TTNtv), known to be a common cause of DCM but surprisingly also present in ∼1% of the general population (83). Mass univariate analyses, adjusted for multiple clinical variables and multiple testing, were carried out at over 40,000 points of a statistical parametric map of 1,409 healthy volunteers. This identified an association between TTNtv

positive status and eccentric remodeling, indicating a previously unproven physiological effect of these variants in subjects without DCM. A similar phenotyping approach was used by Attard et al. in 312 patients to elucidate the physiological mechanisms that underpinned reported association between certain metabolites and survival in patients with pulmonary hypertension (84). Univariate regression models including clinical, hemodynamic, and metabolic data were fitted at each vertices of a 3D cardiac mesh. These showed coherent associations between 6 metabolites and right ventricular adaptation to pulmonary hypertension as well as showing that wall stress was an independent predictor of all-cause mortality.

ML algorithms have also shown promise in predicting outcomes, such as imaging surrogates of disease or response to treatment, from complex sets of clinical and genetic variables. For example, to predict the presence or absence of coronary plaques on CT coronary angiography, a gradient boosting classifier was trained on a proteomic assay and identified two distinct protein signatures (85). A subset of these was found to outperform generally available clinical characteristics in the prediction of patients with high risk plaques (AUC = 0.79 vs. AUC = 0.65), while a distinct set outperformed clinical variables in predicting absence of coronary disease (AUC = 0.85 vs. AUC = 0.70). In another study, a combination of random forest and neural network methods were used first to identify the most informative subset of clinical and genomic data and then to predict coronary artery calcium (86). Interestingly, the model trained on SNP data only was highly predictive (AUC = 0.85), and better than models trained on clinical data (AUC = 0.61) and on a combination of genomic and clinical data (AUC = 0.83). Further validation experiments in patients with less severe coronary artery calcium showed poor predictive accuracy suggesting that the models' predictive value is limited to a range of (high) coronary calcium or that the models do not generalize well in the broader population. Schmitz et al. investigated the performance of 15 different supervised machine learning algorithms in predicting positive cardiac remodeling in patients that underwent cardiac resynchronization therapy (CRT) from clinical and genomic data (87). Several of the approaches demonstrated clear overfitting (accuracy ∼100%), while the algorithm that was identified as the most useful had a fair performance (accuracy = 83%) in addition to high transparency (predictive features easily identified).

Novel deep learning methods are also starting to make an impact in the imaging-genetics field by enabling unprecedented high-throughput image analysis. For example, DL methods have been able to achieve fully automated analysis of CMRs with a performance that is similar to human experts (88) and permitted the rapid segmentation of 17,000 CMRs that were then used in a GWAS (89). This identified multiple genetic loci and several candidate genes associated with LV remodeling, and enabled the computing of a polygenic risk score (PRS) that was predictive of heart failure in a validation sample of nearly 230,000 subjects (odds ratio 1.41, 95% CI 1.26 – 1.58, for the top quintile vs. the bottom quintile of the LV end-systolic volume).

While the use of AI in cardiovascular imaging-genetics has great potential, the limitations and challenges of AI in genetics (90) and imaging (91) are further amplified by combining these very large data. To date, no methodological approaches have been able to include whole-genome and high-resolution wholeheart phenotypes, without requiring extensive dimensionality reduction, filtering and/or feature selection, possibly introducing errors or biases to the input data. Even when this challenge is dealt with, multiple testing correction will continue to be problematic, with the potential for false positive findings likely to only be reliably addressed with replication studies. In AI imaging-genetics, no single method is universally applicable, and the choice of whether and how to use ML or DL approaches will remain task, researcher and population specific, creating difficulties in the pooling of data and meta-analyses. It should not be forgotten that conventional analysis remains valid and has advantages when data are scarce or if the aim is to assess statistical significance, which is currently difficult using deep learning methods. Issues related to the lack of interpretability ("black box") of some ML algorithms are less of an issue in imaging analysis, where accuracy of analysis can be visually verified, but very relevant to integrated imaginggenetics analysis or risk prediction, where identifying and explaining the features driving the algorithm's output can be virtually impossible. The tendency to over-fit models to training datasets risks reduction in the performance of the model when applied to new populations. These problems are likely to be exacerbated if new test datasets include subjects with differing genetic or physiological backgrounds, data were acquired using different technical conditions (e.g., different scanners or different genotyping batches) or if the quality of data acquired in the research setting significantly differs from real world data sets. Finally, issues regarding privacy, ownership, and consent over vast amounts of genetic and imaging data and legal and ethical considerations for clinicians using integrated imaging-genetics algorithms will become an ever more relevant topic of debate.

Although the application of AI to imaging genetics-research is still new, these promising methods and findings warrant further extensive validation in independent populations. Fully integrated, end-to-end, imaging-genetics DL approaches are theoretically extremely attractive but as yet untested. To confidently implement AI methods in research and clinical practice, challenges regarding standardization of data acquisition and algorithm development and reporting still need to be overcome. Initiatives such as adapting the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) recommendations (92) to machine learning research [TRIPOD-ML (93)] are very much welcome. Ultimately, the additive value of AI-driven decision making may

REFERENCES

require robust multi-center studies and randomized controlled trials (94, 95).

### FUTURE PERSPECTIVES

The development of body imaging, the elucidation of inheritance and genetics and the application of statistics to medicine were some of the most important medical developments of the past millennium (96). AI now provides an unrivaled ability to integrate these three aspects in imaging-genetics studies of unprecedented scale and complexity. The increasing variety and capabilities of ML tools at the disposal of researchers provide a powerful platform to agnostically revisit classical definitions of disease, to more accurately predict outcomes and to vastly improve our understanding of the genetic and environmental underpinnings of cardiovascular health and pathology. ML approaches will play an increasing role in every field of cardiovascular research, from genomic discovery and deep phenotyping, to mechanistic studies and drug development. Concerted efforts to improve AI study design, reporting, and collaborative validation will greatly contribute to deliver on the great promise of AI and ultimately improve patient care.

### AUTHOR CONTRIBUTIONS

AM, TD, and DO'R contributed to the content and writing of this manuscript.

### FUNDING

AM, TD, and DO'R research was supported by the British Heart Foundation (RG/19/6/34387, NH/17/1/32725, and RE/13/4/30184); the National Institute for Health Research Biomedical Research Centre based at Imperial College Healthcare NHS Trust and Imperial College London; and the Medical Research Council, UK. AM acknowledges additional support from the Academy of Medical Sciences (SGL015/1006) and a Mason Medical Research Trust grant.

### ACKNOWLEDGMENTS

The authors would like to thank Drs. Wenjia Bai and Carlo Biffi (Department of Computing, Imperial College London, London, UK) for their critical review of this article.

a critical review of progress and potential. Biol Psychiatry. (2017) 82:165–75. doi: 10.1016/j.biopsych.2016.12.030

**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2020 de Marvao, Dawes and O'Regan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Image-Based Cardiac Diagnosis With Machine Learning: A Review

Carlos Martin-Isla1 \*, Victor M. Campello1 , Cristian Izquierdo1 , Zahra Raisi-Estabragh2,3 , Bettina Baeßler 4 , Steffen E. Petersen2,3 and Karim Lekadir 1

1 Departament de Matemàtiques & Informàtica, Universitat de Barcelona, Barcelona, Spain, 2 Barts Heart Centre, Barts Health NHS Trust, London, United Kingdom, 3 William Harvey Research Institute, Queen Mary University of London, London, United Kingdom, 4 Department of Diagnostic & Interventional Radiology, University Hospital Zurich, Zurich, Switzerland

Cardiac imaging plays an important role in the diagnosis of cardiovascular disease (CVD). Until now, its role has been limited to visual and quantitative assessment of cardiac structure and function. However, with the advent of big data and machine learning, new opportunities are emerging to build artificial intelligence tools that will directly assist the clinician in the diagnosis of CVDs. This paper presents a thorough review of recent works in this field and provide the reader with a detailed presentation of the machine learning methods that can be further exploited to enable more automated, precise and early diagnosis of most CVDs.

#### Edited by:

Sebastian Kelle, Deutsches Herzzentrum Berlin, Germany

#### Reviewed by:

Joao Bicho Augusto, Barts Heart Centre, United Kingdom John Hoe, MediRad Associates Ltd, Singapore

> \*Correspondence: Carlos Martin-Isla<EMAIL_ADDRESS>
#### Specialty section:

This article was submitted to Cardiovascular Imaging, a section of the journal Frontiers in Cardiovascular Medicine

> Received: 01 November 2019 Accepted: 06 January 2020 Published: 24 January 2020

#### Citation: