2.1.3 Procedure
throat
SAID
Each session began with two screening questions. Participants could continue only if they said they were eligible to vote in the US and said no to the question, “Do you know a lot about politics in Australia?” They were then given basic instructions about the experiment, given information about how they could contact the experimenters with any questions or concerns they might have, and asked, in accordance with HHS rules, for their consent to participate. Participants were then asked a series of demographic questions, including questions about their political leanings, and then asked, on 10-point scales from “Not familiar at all” to “Very familiar,” how familiar they were with each of two Australian political candidates: Scott Morrison and Bill Shorten, as we noted above.Participants were then given a short paragraph about each candidate (see Participants were now given a task to complete: They would be given an opportunity to scroll through a series of tweets in order to gather information to help them decide which of the two candidates “will do a better job of protecting Australia from foreign threats.” They were instructed to scroll through “all the tweets” before making up their minds. See On the next screen, participants saw a mobile-phone image displaying a series of tweets (Participants had been randomly assigned to one of three groups: Pro-Shorten, Pro-Morrison, or Control. People in all three groups had access to the same randomized sequence of 30 tweets authored by 30 different fictitious people; all the tweets were composed by the experimenters. Five of the tweets portrayed Bill Shorten in a positive light as a protector of Australia; five portrayed Scott Morrison in this same light; and the other 20 tweets simply commented on various ways of protecting Australia without referring to the candidates. All contained the hashtag #protectAustralia.For participants in the Pro-Shorten and Pro-Morrison groups, 5 more tweets were added to the original sequence of 30 tweets, so users in these two “bias groups” had access to 35 tweets in all. In the context of this experiment, the five additional tweets should be considered TMs. These were messages presumably coming not from Twitter users but from the Twitter company itself. In real Twitter feeds, we estimate that the Twitter company typically inserts its own tweets roughly 20% of the time. Sometimes these messages are advertisements; sometimes they include links to breaking news stories; and, close to Election Day, they might include reminders to vote or to register to vote (see In the two bias groups, the TMs appeared in positions 2, 7, 12, 25, and 31 in the sequence of 35 tweets available in their Twitter feeds. The ordering and positions of the TMs were not varied. The only difference between the content seen by members of the Control group and members of the two bias groups was that people in the latter groups saw the five TMs, whereas people in the Control group did not. In addition, the only difference between the TMs seen by members of the Pro-Morrison bias group and members of the Pro-Shorten bias group was that people in the former group saw TMs favoring Morrison, whereas people in the latter group saw those same TMs with the names switched, so that they now favored Shorten.For example, in the Pro-Morrison group, participants saw either strongly negative messages about Shorten such as “Bill Shorten charged with driving under the influence while vacationing in Adelaide,” or strongly positive messages about Morrison, such as “Scott Morrison awarded an honorary doctorate from the University of Melbourne, in recognition for his humanitarian efforts during the Australian wildfires.” As noted above, the TMs were identical in the pro-Shorten group, except that the candidate’s name was changed to his opponent’s name (see The Continue button in the lower-right corner of the web page was inactive for the first 30 seconds of the Twitter session, thus encouraging participants to spend some time reading tweets. If they clicked on the button before it was active, a message was displayed reading, “You have spent too little time reading this page. Please read more.” Also to encourage reading, a message appeared at the top of the page above the mobile phone image reading, “Scroll through the tweets below. You will need to spend some time viewing the tweets before you can continue to the next page.” A maximum of 5 minutes was allowed for examining the tweets in the Twitter feed.On the surface, it might not be obvious how sending different tweets to people in different groups qualifies as targeted messaging. That we are indeed targeting our messages should be clearer if one imagines combining all of our participants into one large group. Now imagine that we divide the group up into three subgroups, perhaps based on certain demographic characteristics (such as income, gender, or political leaning). We now target the members of two of those subgroups with tweets favoring, say, one political candidate; we send no such tweets to the third subgroup. This is how targeted messaging works on any platform, and the message can contain almost any content: a prompt to vote or to register to vote, a reminder to buy one’s loved one a gift on Valentine’s Day, or an advertisement for throat lozenges. The message is targeted as long as it is deliberately being sent to one group and not another, and one knows the targeting has been effective if one can detect predictable changes in the behavior of the targeted group.Following the Twitter experience, participants were again asked a series of questions. The first question was related to the task that had been assigned earlier. “Based on your Twitter search, which candidate, if either, do you think will do a better job of protecting Australia from foreign threats?” (11-point scale from 5 to 0 to 5). Following the “task” question, participants were again asked the six opinion questions and the two voting questions they had been asked before they saw the Twitter feed (see above).Next, participants were asked whether any of the content they had seen in the Twitter feed “bothered” them in any way. They could reply yes or no, and then they could explain their answer by typing freely in a text box. This is a conservative way of determining whether people perceived any bias in the content they had seen–especially bias in the TMs that had been shown to people in the two bias groups. We could not ask people directly about their awareness of bias because leading questions of that sort often produce misleading answers [The session concluded with general information about the goals of the research and about how people could withdraw their data from the study if they wished to do so. No participants chose to withdraw their data from any of the four experiments in the present study.