Strengths and Limitations
BLIND
This randomized clinical trial had several strengths, including use of a scalable intervention developed with user-centered design that was targeted to physicians and patients and evaluated using mixed methods. It also had limitations. For example, we could not blind participants to study group. Participants included English-speaking patients with email access in an academic medical setting and were largely well-educated and White, limiting generalizability beyond this population. Only 18% of recruited patients were enrolled, which may have resulted in a patient sample that was more engaged than the general population. Results may be limited by recall bias. We were not able to capture how often physicians accessed reference materials since we supplied them via multiple modalities (including email attachments), though qualitative results suggest at least some did review them carefully. Additionally, the intervention may have caused some physicians to not offer low-value tests in the first place, which would preclude our primary outcome of conversation quality (in support of this, fewer intervention group patients reported discussing a test with their physician and fewer potentially low-value tests were ordered during their study visits, though these differences were not statistically significant).