Strengths and Limitations
coronavirus disease 2019
CORONAVIRUS DISEASE 2019, RECRUITMENT
One innovation employed was those trial procedures, other than manual follow-up and saliva sample posting, were fully automated once setup, meaning study running costs were relatively low. Approximately half of the participants did not require any human involvement at any stage of their trial involvement and for those that did; it was mainly manual telephone follow-up. As the automated procedures were successful, few changes would be needed when running a definitive evaluation, further reducing resource need and the risk of recruitment and measurement issues.Further strengths were applying robust randomization and intervention delivery fidelity, publishing protocols, and making the statistical and health economic analysis plan publicly accessible prior to analysis, embracing key principles of open science.A key limitation was the poor return rate of saliva samples and that this was imbalanced between arms. It is possible the overall response was affected by the coronavirus disease 2019 pandemic, for example, more limited access to postal services due to changes in movement and time spent outside of the home or hesitation to provide a sample. The low incentive of £5 for a returned sample is likely to have been a factor. Similar trials have provided higher incentives for saliva returns and achieved higher response rates, including digital cessation intervention trials of online smokers (£20 incentive, 75% response rate; personal communication)An additional limitation was that the oldest participant was 61 years old and so our sample did not include those in older age groups. This likely reflects the online advertising approach adopted, although the mean age of participants in the present trial aligns closely with digital cessation trials that recruited offline.A further limitation was that this trial was undertaken during an unprecedented time of changed habits and routines due to the coronavirus disease 2019 pandemic. Participants reported reduced movement due to lockdown and similar measures to reduce movement outside of the home. It is likely this affected the exposure and time spent in different smoking locations and consequently the app’s ability to deliver context-specific support. Furthermore, the app’s use of geofencing to specify locations has potential for measurement error which can cause false negatives, where no support message is sent because of uncertainty as to whether an individual is within a geofence, due to poor location accuracy.