DISCUSSION
Psychiatric comorbidity
In this yearlong pragmatic trial composed of tens of thousands of VA patients and their appointments in primary care and mental health, we found no effect of incorporating nudges in appointment reminder letters on measures relevant to outpatient appointment attendance, specifically the rate of missed appointments and rate of canceled appointments. The lack of effectiveness of incorporating nudges was seen consistently in comparing all four intervention arms to the control, in comparing each individual intervention arm to the control arm, and in both primary care and mental health, the two largest clinic types in VA. While this was a negative trial, we believe our findings are revealing and important to disseminate, particularly in light of recent analyses suggesting substantial publication bias in the realm of nudge interventions, and perhaps even null effects after accounting for this bias.As a pragmatic trial, our findings may have been impacted by several real-world issues within the VA healthcare system. First, a number of veterans were in more than one study arm, particularly among patients in mental health. Psychiatric comorbidity among VA-using veterans is common, and this likely contributed to patients being seen by multiple mental health providers.Our trial was also susceptible to a variety of “on-the-ground” changes in healthcare operations that occurred over the course of the trial. COVID-related impacts included a large-scale switch to virtual appointments, and processes related to this change may have diminished relevance and impact of the trial’s different patterns of outreach to patients. For instance, during the trial it became standard practice for providers to call a patient if they did not log into a virtual appointment, thereby potentially creating a new intervention and obviating a missed appointment. We were also unable to quantify the fidelity of the intervention and cannot guarantee that the offices preparing and sending the appointment reminder letters consistently followed standard procedures.While our pragmatic trial was large, it was nonetheless limited to a single VA healthcare system among patients who were primarily White, male, and older adults. Findings could differ in other patient populations.It is also plausible that no intervention focused simply on appointment reminders may be able to create significant change in appointment attendance. Financial incentives—whether framed as a gain or loss—can be used to influence health behaviors, though their cost-effectiveness would need to be considered.In conclusion, in this large pragmatic trial in primary care and mental health clinics, we found appointment reminder letters incorporating brief behavioral nudges did not affect appointment attendance. Attempts to reduce missed appointments or increase cancelation of appointments that are no longer necessary likely require more complex or intensive interventions.
