2.8. Statistical Methods and Sample Size Calculation
In this study, significance testing was conducted using a 2-sided test, and the significance level was set at 0.05. Values are summarized by means and 95% confidence intervals for continuous variables and counts for categorical variables. The number of correct diagnoses was compared between groups by Fisher’s exact test. The required time for a medical interview and the number of questions asked by the resident physicians were analyzed by the Mann–Whitney U test. All statistical analyses were performed using SAS software (version 9.4, SAS Institute Inc., Cary, NC, USA). The sample size was calculated as follows. Based on previous cases, the correct answer rate with assistance was estimated to be 90%. Since the correct diagnosis rate for trained physicians is generally 80%, the correct response rate for the control group without artificial intelligence assistance was set at 70%, with a difference of 20%. Even with the low correct answer rate of the resident physicians, the difference between the correct answer rate when they received artificial intelligence assistance and the control group was assumed to always be 20%. Under this assumption, the sample sizes were calculated for the control group at 30, 40, 50, 60, and 70% correct, respectively, and 93, 97, 93, 82, and 62 persons were needed for each group. Thus, the case needing the largest number of residents required 97, which means that 97 residents per group were needed. Since this was an exploratory study, and because of considerations regarding the feasibility of the study, 20 residents with and without artificial intelligence-based support, respectively, were set. The 20 resident physicians were to examine 5 simulated patients with AI support and 5 without AI support for a total of 200 answers. The 200 results were obtained by 20 resident physicians, 10 from each. However, the 10 simulated cases were independent, and the study was designed assuming that the individual results were independent. In addition, since this was an exploratory study, no adjustment for multiplicity was made.