Testing variability in model performance by time and sample size
suicidal behavior
SAID
We also examined the average time that the model was able to predict suicidal behavior in advance of an individual receiving a case-defining diagnosis for suicidal behavior. Setting specificity at 90%, the model identified 37% (Finally, to account for the fact that our MS-specific model was trained on a much smaller set of data, we developed a similar Naive Bayes model on a training set of equal size to that of the MS sample from the general population, and then tested this model on the same validation set of individuals diagnosed with MS as the MS-specific model. The results were an AUC of only 0.66 (compared to 0.77 in the MS model). That being said, we found that the size of the cohort had a very large impact—when building the model using a training set of 