In sea of data, not all numbers are equal

The Washington Post
Published Wednesday, November 15, 2006

WASHINGTON -- When I called my grandmother on Election Day, she said, "I hate pollsters." She's probably not alone as we collectively recover from Nov. 7 and its hysterical run-up.

People paying attention to this election -- and even those trying not to -- got their fill of polls. And in many cases, the numbers unhelpfully painted starkly different pictures of this year's campaigns.

"Poll Puts Maryland Democrats in the Lead," The Washington Post reported in late October on the state's races for governor and U.S. senator, giving the Democratic candidates sizable advantages. Only hours before that poll came out, a slew of other polls showed the two campaigns much closer, leading the respected Cook Political Report to add the Senate race to its list of tossups.

In Maryland and nationwide, candidates trailing in one poll could point to another poll in which the race was "too close to call" or even had the opposite estimate. One can hardly blame a trailing candidate for using implausible polls as a pick-me-up, but the effect is universally confusing.

And that's too bad. Polling is central both to reporting and to understanding who we are, why we vote and what we expect out of our political system.

Before last week's election, President Bush derided election "prognosticators," but no one would be so blithe as to dismiss public attitudes, as communicated via ballots or reliable polls.

For the media, public opinion data help flesh out and add context to reporting and expert analyses. Survey data help stories rise above anecdote and provide a broader perspective.

That's why it's tempting to use numbers in stories and arguments and why it's so essential to recognize that all numbers are not created equal. So, what to do?



These polls were not magically predictive; rather, they captured the main themes of the election that were set months before Nov. 7. Describing those Maryland contests as tight races in a deep-blue state, in what national pre-election polls rightly showed to be a Democratic year, misled election-watchers and voters, although cable news networks welcomed the fodder.

More fundamentally, averaging polls encourages the already excessive attention paid to horse-race numbers. Pre-election polls are not meant to be crystal balls. Putting a number on the status of the race is a necessary part of pre-election polls, but much is lost if it's the only one.


This last point exaggerates the disparities among pollsters. But there are differences among pollsters, and they matter.





To the contrary, an ancillary burden is on survey research organizations to openly disclose how and why they ask the questions they do, so people can judge for themselves whether to believe the results.

Pollsters should disclose the sponsorship of their polls, provide a detailed statement on how they conduct their research and reveal their full questionnaires. Few will be interested in reading all this, but it should be readily available. So many polls are part of political or public relations campaigns that it's particularly important to be able to check for signs of bias.

News organizations should carefully review all such information before putting their names on polling data by publishing it. Most people have finely tuned nonsense filters, many have changed their computer settings to avoid countless e-mail offers, and it's high time for the news media to set up poll filters.

Grandma has a nice Southern expression for the current alternative, something about a pig and lipstick. And as one of the aggrieved pollsters she derided last week, I'll try to leave you alone for a while so we can recover from any lingering poll fatigue.