Summary of Mode Differences QUESTIONS At the extremes, four
In general, we found little evidence that cognitive processes of these sorts created mode differences in responses. That may reflect the fact that the questions chosen for this study are drawn from well-tested items designed for telephone surveys, and thus do not reflect the kinds of burdensome items that have previously been shown to create mode effects. It’s also possible that the panelists, having participated in one large telephone survey with Pew Research Center (the polarization study that was used to recruit the American Trends Panel) and – for the vast majority – at least one previous panel wave, are savvier survey participants than others and thus are less vulnerable to mode effects than a fresh cross-sectional sample would be.
Responses to all but four of the 60 items showed at least some numerical difference by mode, and the median difference across all items was 5 percentage points. The largest difference was 18 points, and there were eight items with differences of at least 10 points. But most of the 24 non-zero differences smaller than 5 percentage points are not statistically significant, and thus could have occurred by chance as a result of sampling error.
Summary of Mode Differences  | QUESTIONS  | At the extremes, four  | questions had zero mode  | difference, -point while one had  | an 18 -point difference  | 01234 5  | 10  | 15  | 18  | % point difference between phone and Web  | Source: American Trends Panel Wave 5, Survey conducted  | Jul. 7-Aug. 4 2014  | PEW RESEARCH CENTER 