Discussion
In this trial of board-certified cardiologists adjudicating clinical transthoracic echocardiographic exams, AI-guided initial evaluation of LVEF was found to be non-inferior and even superior to sonographer-guided initial evaluation. After blinded review of AI versus sonographer-guided LVEF assessment, cardiologists were less likely to substantially change the LVEF assessment for their final report with initial AI assessment. Furthermore, the AI-guided assessment took less time for cardiologists to overread and was more consistent with cardiologist assessment from the previous clinical report. Although not the first trial of AI technology in clinical cardiologyIn addition to prospectively evaluating the impact of AI in a clinical trial, our study represents the largest test–retest assessment of clinician variability in assessing LVEF to date. The degree of human variability between repeated LVEF assessments in our study is consistent with previous studiesNotwithstanding tremendous interest in AI technologies, there have been few prospective trials evaluating their efficacy and effect on clinician assessments. Important clinical trials of AI technology have already shown the efficaciousness of AI in cardiologyTo enable effective blinding, we implemented a single cardiac cycle annotation workflow representative of many real-world high-volume echocardiography laboratories. Despite this framework, there was a small signal for cardiologists to be more likely to be correct than incorrect in guessing the agent of initial assessment. However, the blinding index is within the range typically described as good blinding, and regardless of whether the cardiologist thought the initial agent was AI, sonographer or uncertain, the results trended towards improved performance in the AI arm. Our findings of non-inferiority and even superiority of initial AI assessment are encouraging given that AI assessment reduces the time and effort required of the tedious manual processing that is typically required by routine clinical workflows. Given these promising results, further developments of AI could eventually facilitate additional workflows that are required for conducting comprehensive cardiac assessments in routine clinical practice and in accordance with guideline recommendationsSeveral limitations of our trial should be mentioned. First, our study was single centre, reflecting the demographics and clinical practices of a particular population. Nevertheless, the AI model was trained on example images from another centre and the clinical trial was performed as prospective external validation, suggesting generalizability of the AI techniques and workflow. Second, the study was not powered to assess long-term outcomes based on differences in LVEF assessment. Although the results were consistent across subgroups, further analyses are needed to evaluate the impact of video selection, frame selection and intra-provider variability. Third, this trial used previously acquired echocardiogram studies, and although prospectively evaluated by sonographers and cardiologists, there can be bias when a different sonographer than the scanning sonographer interprets the images. Last, consistent with findings from most AI studies, we found model performance improvement scales with the number of training examples. Thus, we anticipate that future studies could improve on the AI performance that we observed in the current study by implementing AI models developed based on an even greater number of training examples derived from a broad and diverse cohort of patients. Of note, this clinical trial utilized an AI model entirely trained from an independent site, representing external validation of the model. Effective deployment of AI models in cardiology clinical practice will require additional regulatory oversight, adoption and appropriate use by clinicians, and functional integration with clinical systems, all of which need to be carefully considered and further studied.In summary, we found that an AI-guided workflow for the initial assessment of cardiac function in echocardiography was non-inferior and even superior to the initial assessment by the sonographer. Cardiologists required less time, substantially changed the initial assessment less frequently and were more consistent with previous clinical assessments by the cardiologist when using an AI-guided workflow. This finding was consistent across subgroups of different demographic and imaging characteristics. In the context of an ongoing need for precision phenotyping, our trial results suggest that AI tools can improve efficacy as well as efficiency in assessing cardiac function. Next steps include studying the effect of AI guidance on cardiac function assessment across multiple centres.