Conclusions
In this diagnostic study evaluating the use of deep-learning NLP to measure EHR-documented goals-of-care discussions, we measured the primary outcome of a large pragmatic clinical trial using NLP-screened human abstraction with acceptable sensitivity and substantial savings in abstractor-hours. Our experience demonstrated that NLP may facilitate clinical research studies that would otherwise be infeasible due to the costs of manual medical record abstraction. Misclassification-adjusted power calculations quantified power loss from NLP-related misclassification, suggesting that incorporation of this approach into the design of future studies that use NLP to measure outcomes would be beneficial.