In defence of (some) pollsters

Following the result of the EU Referendum, numerous articles were written about the ‘losers’ of the vote and, in some, pollsters featured prominently. Following the General Election last year, pollsters were, rightly, attacked for failing to see the Conservative majority that unfolded. This led to lots of soul searching, an independent inquiry and many sleepless night. However, the charge that pollsters missed the outcome this time rings hollow. Some polls foretold the result throughout the campaign but were largely ignored by a sceptical media intent on trumpeting only those polls that showed what they thought would happen.

Populus, the ‘Remain’ campaign’s pollsters commissioned a piece of work looking at why telephone and online polls were showing consistently different results. Phone polls had shown remain well ahead whilst online polls showed a tight race with Leave ahead more often than Remain. This analysis concluded that the true state of public opinion was likely between the two modes but, crucially, that phone polls were probably more accurate on this occasion. This piece of work was reported on Newsnight and most articles written following this made reference to phone polling being the more accurate guide.

For our part at YouGov, our analysis, published weeks before the referendum, highlighted the issues we saw with the phone pollsters. It was evident from our research that too many graduates were willing to take phone surveys and, with education the strongest driver of support for Remain, the phone polls were likely underplaying the likelihood that Leave would prevail.

There are important questions to be asked as to why the media latched onto this narrative given phone and online polls called the referendum campaign completely differently. If only online polls were considered during the referendum campaign, the shock of a Brexit would have been reduced with the writing on the wall for some time.

eu_campaign_polls.png

During the campaign period, 33 polls showed Remain ahead with 32 showing Leave in the lead. Of the 32 polls showing Leave ahead, just six were carried on the phone.

As the US data guru Nate Silver notes, the Brexit polls showed a toss-up between Remain and Leave and the question we should be asking is why so many in the media ignored evidence to the contrary.

nate_silver.png

Much of the discussion following the polling miss in 2015 focussed on the fact that the polls showed a hung parliament and this dominated the media coverage; there was too much focus on who would make up a coalition and not enough scrutiny of the Conservative manifesto and their promises. The same could be true for the EU Referendum, but it isn’t the polls who are to blame. By putting their faith in the phone polls, the media spent a significant proportion of their time scrutinising David Cameron’s position in the event of a Remain victory and not enough time discussing the real affect a Leave vote would have on the country. Whilst the phone pollsters have some significant questions to answer following their performance calling the EU Referendum, “the pollsters got it wrong again” will simply not cut it this time. 

Argue the issues with like minded people by leaving a comment below or joining the discussion here


Be the first to comment

Please check your e-mail for a link to activate your account.