Methods
This research was reviewed by the Institutional Review Board at the University of Pennsylvania, which deemed it exempt and waived documentation of participants’ consent. The study design used a randomized, controlled experiment to assign each participant to one of four conditions. Although the primary goal was to compare responses from each study arm, we also consider responses within subgroups, which could generate hypotheses that can be tested in future studies. The experiment was designed to examine if, compared to a control condition, a relatively large incentive, small financial incentive, or a message about vaccine safety and efficacy (which is a typical educational approach to interventions) was associated with an increase or decrease in the percent reporting that they would want to vaccinate against COVID-19.Study enrollment was conducted through Prolific (With any experiment, data quality can be reduced if an individual participates more times than researchers intended. We relied on automated procedures that make it difficult for the same individual to participate in the experiment more than once [We recruited a theory-based sample (rather than using probability-based sampling) based on income and race/ethnicity. We did not enroll a nationally representative sample for two reasons. First, vaccine uptake is not evenly distributed across the U.S. population [Using Prolific’s automated features, we screened potential participants by their self-reported race/ethnicity and household income. We enrolled those with household incomes less than $80,000—a group that accounts for more than 50% of the U.S. population and that has lower rates of vaccination compared to wealthier Americans [When this experiment was conducted in 2021, the FDA had announced emergency authorization of the vaccines, but vaccines were not widely available [Using an algorithm that applied a 1:1:1:1 allocation ratio, we randomly assigned unvaccinated individuals to one of four groups. In one of the four groups, participants were notified about a potential government policy to pay $1,000 for COVID-19 vaccination. In a second group, the payment was listed as $200. In the third group, participants received information regarding COVID-19 vaccines’ safety and efficacy (i.e., without any payment policy information). A fourth group served as a control condition and received a general message, with no reference to COVID-19 or vaccination. To control respondent burden, the questionnaire text for each arm was similar in length.Following exposure to the vaccine policy message, respondents were asked about their interest in vaccination. For instance, as shown in the Supplement, among those assigned to the $200 financial incentive policy, we asked: Respondents also reported standard demographic variables, such as their age, race, ethnicity, and gender. Socio-economic status was reflected in a measure of educational attainment and their personal financial circumstances. The latter is a subjective measure that is often equally or more relevant to psychological outcomes than an objective measure of income [We collected all data on an external website using the web-based software, Qualtrics. Using STATA 17.0 statistical software, for each of the experiment’s four conditions, we calculated the proportion of respondents who reported that they wanted to vaccinate. This proportion was reported overall, for the primary analysis (as noted in preregistration) and by subgroups for exploratory analyses. The subgroups were specified by age, gender, race/ethnicity, education (college graduate vs. non-graduate), and financial circumstances.Overall and within subgroups, we assessed whether differences in the proportion wanting to vaccinate across the four study arms were statistically different using group ANOVAs at the 95% level of significance. To allow for pairwise comparison, we also estimated Tukey honestly significant differences (HSD) tests overall and within subgroups; results are available in the Supplement. Tukey HSD tests can assess multiple pairwise differences for the probability of a Type 1 error [