Why there isn’t a British Nate Silver

by Stephen Tall on November 8, 2012

A skim-read of Wednesday morning’s headlines might have left folk confused as to who had been declared the victor of the US presidential election: Barack Obama or Nate Silver.

For those who don’t know Nate Silver, he’s the analytical guru behind the FiveThirtyEight blog (named after the number of electoral college votes), now housed at the New York Times, which scrutinises and filters opinion polls. He first rose to prominence four years ago after predicting the winner in 49 of the 50 US states during the Obama-McCain presidential contest, and re-inforced his reputation two years later by forecasting correctly all 35 senate races.

His cred has now reached near-messianic status. First, because yet again his methodology has been validated — if Florida is eventually declared for Obama, Nate will have another clean sweep of predictions. Secondly, because he has consistently called-out the media for portraying the race as tied despite the high likelihood that Obama has maintained a solid electoral college majority throughout the year. And thirdly, because of the personal insults that have been flung in his direction over the past few weeks by Romney supporters aggrieved that Nate didn’t have better news for their guy.

I’m a huge fan of Nate Silver’s forensic approach and FiveThirtyEight has been my first port of call to find out how the US presidential race is actually going, rather than through the mainstream media’s unfocused lens. But Daniel Engber makes a fair comment on Slate.com when he points out: “Nate Silver didn’t nail it; the pollsters did.”

And that is, I suspect, the single biggest reason this country hasn’t yet, and is unlikely to, produce our own version of Nate Silver — because we don’t have the necessary quality of polls and because we don’t have a media interested in actually finding out what’s going to happen. Let me unwrap those two assertions:

1) Absence of local polling

The reason most commentators could get away with their blather that the US presidential race was tied was because they reported the headline national numbers. These did indeed show Obama and Romney to be essentially neck-and-neck, and that’s how the national result has finished. But averages conceal as much as they reveal. What Nate Silver (and others psephologists such as RealClearPolitics and Talking Point Memo) highlighted was the data from state polls, in particular the ‘toss-up’ states which would decide the next occupant of the White House.

The same mistake that Nate Silver’s decriers made is almost universally made day-in-day-out by British pundits using national polls to project the next general election result. Ironically this includes many commentators who’ve praised Nate Silver’s approach in the US and yet disregard its applicability in the UK (yes, I’m looking at you John Rentoul).

Everyone knows that results differ across the UK: that what happens in Cornwall won’t be repeated in London or Scotland or the East Midlands etc. Yet when it comes to projecting the results of the next general election, the media is all too happy to shove a load of polls in the blender and hope it approximates to reality.

The problem is highlighted by Nate Silver himself. He attempted to devise a model for forecasting the last UK general election based on the polls. The final projection showed the Lib Dems in second place and winning 120 seats. Does this show Nate’s voodoo is a load of hooey? No, it shows that the British focus on national polling is a poor basis for projections of numbers of MPs. And it is also fair to point out that Nate Silver 1) inserted many caveats into his prediction, and 2) came up with a range of projections, the final one of which (scroll down to the bottom of this page) wasn’t so very far off.

2) Absence of media interest in accuracy

So if we accept that national polling is a deeply imperfect guide to projecting what the House of Commons will look like after the next election, it’s fair to ask the question why doesn’t the media — why don’t the polling companies — commission the kind of research that is needed to make accurate forecasts? It’s a question easily answered with another question: what’s in it for them?

I’m sure the polling companies themselves would be very happy to undertake more regional polling, to find out if the results in the North-West will differ from those in East Anglia. But they’ll need paying, not least because political opinion polling is only a very small part of their business (albeit the most publicly visible). And as there isn’t a strong, profitable regional media it doesn’t happen. There have been sporadic attempts to do it: PoliticsHome and The Sun have both commissioned extensive polls of marginal seats, but these are special one-offs so no reliable trend emerges that allows a Nate Silver-style calibration.

The blunt reality is that the news media craves excitement more than it hungers for truth. It is much cheaper and easier to commission a monthly survey and then inflate the results way beyond what the data should allow. We’ve all seen the kinds of headlines newspapers revel in — ‘Poll blow to Tories as support plunges 1%’, ‘Labour to win 100+ majority says latest exclusive poll’ — and yet journalists continue to write them even though they know deep down how flimsy the evidence is.

Finally

I’ve read lots of adulation of Nate Silver in the British media in the past 24 hours. I wonder if any of those journalists who’ve penned those articles have thought, even for a moment: I wish I had the confidence to write about polls with the same kind of rigour he does. I’m not holding my breath.

* Stephen Tall is Co-Editor of Liberal Democrat Voice, a Research Associate for the liberal think-tank CentreForum, and also writes at his own site, The Collected Stephen Tall.

15 comments

New from me > Why there isn’t a British Nate Silver http://t.co/UK7V5Yzk

by Stephen Tall on November 8, 2012 at 9:29 am. Reply #

New from me > Why there isn’t a British Nate Silver http://t.co/YmQo2B5r

by Stephen Tall on November 8, 2012 at 12:29 pm. Reply #

New from me > Why there isn’t a British Nate Silver http://t.co/YmQo2B5r

by Cameron Yarde Jnr on November 8, 2012 at 12:30 pm. Reply #

Why there isn’t a British Nate Silver http://t.co/TqvwXy65

by BexhillBattleLibDems on November 8, 2012 at 12:33 pm. Reply #

Yes.

by Louise Shaw on November 8, 2012 at 2:26 pm. Reply #

Why there isn’t a British Nate Silver | Stephen Tall http://t.co/fIovWvvw via @stephentall

by Louise Shaw on November 8, 2012 at 2:58 pm. Reply #

Why there isn’t a British Nate Silver | Stephen Tall http://t.co/fIovWvvw via @stephentall

by RC Richards on November 8, 2012 at 3:11 pm. Reply #

Why there isn’t a British Nate Silver | Stephen Tall http://t.co/fIovWvvw via @stephentall

by Christopher Zorn on November 8, 2012 at 3:18 pm. Reply #

Why there isn’t a British Nate Silver | Stephen Tall http://t.co/fIovWvvw via @stephentall

by Jay Ulfelder on November 8, 2012 at 3:21 pm. Reply #

New from me > Why there isn’t a British Nate Silver http://t.co/TDXPrHD6

by Stephen Tall on November 8, 2012 at 3:29 pm. Reply #

@robfordmancs: Some people seem to have missed the Polling Observatory 2010 election forecasts =>
http://t.co/UhyoyO3u

by Will Jennings on November 8, 2012 at 3:51 pm. Reply #

Nate Silver wasn’t the only one offering forecasts in the UK in 2010. Our model did rather better despite the lack of local polling (which as you rightly point out would make things much easier):

http://www.politicshome.com/uk/the_poll_centre.html

by Rob Ford on November 8, 2012 at 4:10 pm. Reply #

New from me > Why there isn’t a British Nate Silver http://t.co/TDXPrHD6

by Eilidh on November 8, 2012 at 5:13 pm. Reply #

[...] you don’t have to travel far in the data landscape to start running into trouble. In a piece posted yesterday, Stephen Tall asks rhetorically why there isn’t a British Nate Silver and then explains that [...]

by It’s Not Just The Math « Dart-Throwing Chimp on November 9, 2012 at 2:10 pm. Reply #

[...] Nate’s confident prediction was acclaimed here in the UK. Yet the lessons for us here are now routinely ignored. Just as the US decides its President through an electoral college, we decide our government through electing constituency MPs. The only way to work out who’s actually most likely to form the next government is to undertake more regional polling and then to extrapolate from that the likely number of MPs for each party, while also weighting for other facts such as incumbency boosts (which disproportionately aid the Lib Dems and first-term MPs). But, as I wrote last November: [...]

by Opinion polls yadda yadda. OR “Does Nate Silver mean nothing to you? Did he write in vain?” on May 21, 2013 at 8:49 am. Reply #

Leave your comment

Required.

Required. Not published.

If you have one.



You might also likeclose
Plugin from the creators of Brindes :: More at Plulz Wordpress Plugins