Discussion
mental disorders
A growing body of research has attempted to leverage the large data resources of EHRs to predict risk of medically-important outcomes, including suicidal behavior [Our predictive model performed well in identifying patients with MS who would subsequently engage in suicidal behavior, with a level of accuracy (AUC = .77) and PPV (1–3%) that is in line with results from other studies that have used machine learning applied to EHR to predict suicidal behavior [Future studies should test whether prediction performance can be similarly improved for other conditions and subpopulations. For instance, it may be that prediction can be improved by generating separate models for: those with vs. without mental disorders, children/adolescents vs. adults, males vs. females, and so on. On balance, these groups can overlap and so methods are needed to control the complexity of the many different lines that can be drawn in the data. In addition, we observed in the current study that generating a predictive model in a subpopulation led to reduced sample size on which to train our model, which in turn limited predictive accuracy. Thus, the increase in accuracy for more specific models must be weighed against the penalty one incurs for building a model in a smaller sample. This trade-off must be weighed in each new analysis and the best solution may depend on whether one values discovery of new risk factors within that subpopulation vs. overall predictive accuracy of the model.These findings must be viewed in light of several key limitations. First, the sample size in our MS-specific model was relatively small, and the observed results would benefit from replication in a larger sample. Second, although our MS-specific model had a stronger AUC than the more general model, there is still much room for improvement in this accuracy. This represents an important direction for future research. Third, the NBC approach used has the benefit of transparency, thus avoiding concerns with “black-box” algorithms [