Overview
RECRUITMENT
This paper presents a case study on participant deception experienced throughout a remotely conducted large randomized controlled trial. Two types of participant deception were detected: automated bots and manual participant deception. Automated bots were identified based on clusters of enrollments very early in the morning with non–United Kingdom–based addresses. A CAPTCHA was added that resolved the issue, and no additional bots were identified. Manual participant deception was discovered due to incorrect or repetitive information provided, including restaurant and hotel contact details instead of home addresses. Monthly data checks to identify any unexpectedly high recruitment periods, suspicious entries, and contacting participants swiftly helped to mitigate the problem, although it required close monitoring throughout recruitment.Considering the circumstances under which recruitment took place, some of our experiences and recommendations may not be applicable when conducting research outside of a pandemic; this is an empirical question. While it is likely the pandemic meant some people were more engaged with web-based research, the highest recruiting day with the most suspicious entries occurred with 312 enrollments on July 29, 2020, when no active recruitment or social media advertising was taking place. Such automated bots have the potential to seriously disrupt recruitment and bias the final results. In our trial, we forecast that the target (n=5562) would have been reached within 3 months rather than the 21 months planned. Without rapid identification, this would have meant that the research budget was being spent compensating bots, with 81.9% (4556/5562) of the final sample estimated to be bots. This would likely have resulted in an underestimation of the effectiveness of the intervention due to the noise from the bot responses.We recommend using a CAPTCHA when setting up a remote trial to deter automated responses. We did not initially use one to ensure the trial was as accessible as possible and so as not to deter “real” participants. However, it is worth noting that a CAPTCHA does not render a survey invulnerable and could still be passed by a manual fraudulent entry [Participant deception occurred on a smaller scale than the bots, accounting for 5% (298/5955) of participants enrolled, and as such, appears to be less of an issue. However, it is also less likely to be detected, and it is also more time intensive for researchers to try to identify suspicious responses based on the contact information provided. Establishing a strict procedure for identification and management meant the research team could act swiftly and deter future attempts. We suspect most of the participant deception was perpetrated by a relatively small number of individuals at different time points attempting to enroll on multiple occasions, as once several of their entries were removed with emails explaining why, the numbers engaging in deception decreased considerably. Without identification and management of this issue, individuals may have continued to submit false entries throughout the study.There were 3 elements of the trial that helped the team to identify manual participant deception. First, participants were followed up 3 times over the course of 6 months, which presented several opportunities for researchers to contact participants and potentially identify any anomalies in contact details. With fewer resources and time spent on follow-up, some of the manual participant deception may have gone undetected. Second, there was no financial incentive offered for enrolling in the study initially, but only for completed follow-ups, so the incentive was delayed, making it less attractive for people seeking an immediate reward. Finally, the vouchers were sent manually by a member of the research team rather than being sent automatically, so there was a time lapse between follow-up completion and compensation being sent and a further opportunity to detect any discrepancies with the information given or similar or duplicate email addresses used before the vouchers were sent.It is worth noting the potential issues with inequalities when creating data management procedures to detect participant deception and the need to strike a balance [