Post-Mortem Ontario: Who's right?

How did the pollsters fare in this election? There are a number of different ways to look at the situation. The first is to see whether the prediction that I made using all the polls' estimates was accurate.  Here is the graph with the election results: It is clear from this graph that the Liberals were globally well predicted by the polls but that, contrary to usual, the PC was overestimated and the NDP underestimated. This a rather rare situation. In the last federal election of 2011, the polls had underestimated the PC by more than 4 points in Ontario.

Who's right?

There is a debate around methods and pollsters. Who fared better? There are known ways to deal with this. One is to calculate the difference between the results and the estimates from the polls. The relative error (results minus estimates) tells us whether a party tended to be over or underestimated by the polls. A positive result means that the party was underestimated by the polls. The absolute error infoorms on the level of the difference between the results and the estimates. The total absolute error gives the synthetic information on the global level of error. It is computed as the sum of the absolute error for the three main parties. I present the results grouped by method (IVR vs Web) for the polls conducted from June 5 in the table at the end of the message.

What the table and my analyses show is:
A total absolute error of 10.8 is rather high. It is however lower than what has been seen in BC2013 and Alberta 2012.

In conclusion,

The surprise with this election is the global underestimation of the NDP and the overestimation of the PC. I do not see how to explain this. The only expected bias, i.e. an overestimation of the NDP vote and an underestimation of the PC vote by the WEB surveys indeed happened. The huge underestimation for the NDP vote by the IVR polls will have to be analyzed by the pollsters. In addition, the "likely voter models" did not seem to perform adequately. It is good that pollsters who used these models indeed published also their figures for eligible voters. Analyst and voters could see that polls allowed for different estimates according to different hypotheses about who is participating in the election. However, the likely voter models do not seem to be "the solution" to bias in polls' estimates. Pollsters who use IVR should perhaps think about introducing cell phone numbers in their samples and see whether it improves their estimates.

The final point to notice -- it has been noticed by many analysts -- is the substantial variation between estimates. This means that the methods right now are not at the level of reliability that we would like to see them.

Publié par Claire Durand à 11 h 33 Publier un commentaire

Twitter: @clairedurand Suivre par courriel - Follow by Email Archives du blogue Qui suis-je Claire Durand
Montréal, Québec, Canada
Claire Durand est professeur titulaire au département de sociologie de l'Université de Montréal. Elle est présidente de WAPOR (World Association for Public Opinion Research). Claire Durand is professor at the department of sociology, Université de Montréal. She is president of WAPOR (World Association for Public Opinion Research)
[null, null, {"document_url": "http://ahlessondages.blogspot.com/2014/06/post-mortem-ontario-whos-right.html", "unformatted_src": "http://3.bp.blogspot.com/-ZUzRJIYBjpE/U5r8jywz5ZI/AAAAAAAAAR8/yX39pTGMXo4/s1600/intvote_Ontario_election1.png", "src": "http://3.bp.blogspot.com/-ZUzRJIYBjpE/U5r8jywz5ZI/AAAAAAAAAR8/yX39pTGMXo4/s1600/intvote_Ontario_election1.png", "formatted_filename": "intvote Ontario election", "rendered_width": 400, "rendered_height": 357}, null, null, null, null, null, null]