Statistical Analysis
diabetes
RETINA, DIABETES
Analysis was restricted to those patients with available hemoglobin A1c (HbA1c) results at baseline (because this was a required part of data collection), known follow-up dates, at least one image at the subsequent screen, and known MOD+ outcomes at the subsequent visit. Patients who did not attend within ±60 days of their appointment date, or attended significantly earlier than expected (<150 days since the baseline visit), were excluded from analysis. Images deemed poor quality by the retina specialist were also excluded from analysis. A CONSORT diagram illustrating participant inclusions and exclusions is presented in The analyses were performed by combining patients across sites. The goal of the analysis was to compare the ranking ability of different approaches to scheduling subsequent screenings:Approach A: Baseline - ordered randomlyApproach B: Baseline - all mild DR then all no DR, random order within each groupApproach C: All mild DR then all no DR, ranked by decreasing HbA1c measurement at baseline within each groupApproach D: Model proposed orderApproach E: Observed screening orderWhereas approach A is a simplified approximation to current practice of following a standardized screening interval, approaches B and C represent clinically informed ordering approaches based on available information (DR grade and HbA1c). Approach D represents our DL model, and approach E is the actual order observed in this study. Further detail on how rankings were combined across sites is provided in the The primary outcome measure was the sensitivity of the model, that is, its ability to rank patients by risk of MOD+ within the first 50% of subsequent screens. This was calculated by dividing the number of patients with MOD+ in the first 50% of rankings by the total number of patients with MOD+ at the subsequent screen.Approaches A and B were considered the baseline approaches for comparison with approaches C, D, and E. To test the superiority of the model's sensitivity, a permutation test (with 10,000 iterations) was performed under the null hypothesis that the ranking by the model is no different from random ranking (approach D vs A). A second permutation test was performed with the null hypothesis of ranking all mild DR then all no DR, random order within each group (approach D vs B). For each, alpha was set at 0.05. We also report statistical tests comparing approach C and approach E versus both stated null hypotheses in the In addition, we conducted a one-sided Mann–Whitney We devised additional ranking approaches using additional baseline variables, namely, age, duration of diabetes, and insulin use. We first restricted this analysis to a subset of patients where all relevant variables were present, yielding 1646 of 1757 patients. For each baseline variable, we devised a ranking scheme which used the baseline grades to rank the patients, and within those two groups (no DR and mild DR), we ranked the patients using the additional baseline variable. Using the progression ground truth, these rankings were then converted into a plot of fraction screened versus sensitivity plots. For a fair comparison, we also reran the previous approaches (A, B, C, D, and E) on this subset of patients. The results are presented in 