The polls: what, and who, to believe?

by Stephen Tall on October 20, 2008

I’ve noted on a couple of previous occasions here on Lib Dem Voice, the sharp divergence which different polling companies’ methodologies produce in the Lib Dems’ ratings. The trouble is that very few political journalists take much notice of such details: how often have you heard sweeping statements from commentators that talk about ‘the polls’, as if they all asked the same questions in the same way, producing the same results?

The trend in recent days had appeared more than usually gloomy for the Lib Dems, with the last seven polls placing the party in the 14-18% range.

Yet tonight the news emerges of the latest ICM poll for the Guardian, which shows the party up at 21%. This comes just one day after YouGov for the Mirror put the Lib Dems at just 14%. That’s an astonishingly wide variance, not all of which can – by any means – be accounted for by the swings associated with party conference season, or the financial turmoil in recent weeks.

To show what a difference it makes to the Lib Dems’ electoral fortunes, let’s feed the two results into Anthony Wells’ seats predictor:

>> YouGov, with Lib Dems at 14%, predicts the party will be reduced to just 20 seats, a devastating drop of 42 MPs.
>> ICM, with the Lib Dems at 21%, predicts the party would end up with 45 MPs, a hefty drop of 17 seats, it’s true, but nowhere near as cataclysmic (and remember also the good reasons to believe that such blanket calculations will under-estimate the number of actual Liberal Democrat seats – see links below).

At least as far as the betting markets are concerned – when the public puts its money where its mouth is – they’re punting on ICM’s view of Lib Dem fortunes proving most accurate. For the rest of us, as ever with the polls – and no matter what our party allegiance – we can choose to believe whichever pollster’s figures offer us the greatest comfort.

Further reading: LDV has previously covered in more detail how YouGov differs from other pollsters, possible explanations for this, reasons to be wary of BPIX and reasons to be wary of Liberal Democrat seat number projections.

Enjoy reading this? Please like and share:

No comments

I was just going to post the details of the ICM poll, but I see you’ve beaten me to it! Funny, that.

by Clegg's Candid Friend on October 20, 2008 at 8:46 pm. Reply #

Come on Tall! You know there’s only one poll we REALLY care about!

by James Graham on October 20, 2008 at 8:55 pm. Reply #

I believe the wide variation is because ICM don’t offer a ‘don’t know/undecided’ option, which is obv. more accurate because you can’t put ‘undecided’ on a ballot (unless you spoil your paper).

by Thomas Hemsley on October 20, 2008 at 9:15 pm. Reply #

Oooooh, Stephen, you TEASE!

by Jennie on October 20, 2008 at 9:24 pm. Reply #

I can has golden dozen??

by Charlotte Gore on October 20, 2008 at 9:46 pm. Reply #

Seriously I can’t remember a time when the Golden Dozen’s not been published before 8:45pm on a Monday!

Conspiracy! Cover up!

by Charlotte Gore on October 20, 2008 at 10:18 pm. Reply #

Mr Tall is just being a big mean meanypants.

by Jennie on October 20, 2008 at 10:20 pm. Reply #

To be honest, I think if I were devoting my time to the Golden Dozen and it was full of people trying to debase it, I probably wouldn’t be so keen to publish it in a hurry.

I suddenly feel like a right meanypants arsehole myself.

by Charlotte Gore on October 20, 2008 at 10:27 pm. Reply #

Returning to the main theme….

I think it’s very egalitarian of the polling companies to produce three polls, one good one for each party 🙂

What worries me to an extent is that all the polling companies have developed a methodology to account for the “silent tories” since the mid 90s. As that trend is unravelling then they need to take account of that but essentially they are guessing at how to account for that (in a very educated and well tested way – but no-one really knows how accurate it will be untill we get to a real election).

The other possibility is that with a highly volatile electorate, minor differences in questioning are having big effects. Still doesn’t really explain why one company shows us up 4 and one down 4 though!

by Hywel Morgan on October 20, 2008 at 10:34 pm. Reply #

Okay, okay, enough already about the Golden Dozen – it’s now compiled and ready to go live …

tomorrow morning.

by Stephen Tall on October 20, 2008 at 10:45 pm. Reply #

The important thing is to look at the trends in all the polls, over a 3 or even a 6 month period. Here, John Curtice’s poll of polls is invaluable.

by Neil Stockley on October 20, 2008 at 10:57 pm. Reply #

Yes. As I posted before, the message of the four previous polls from different companies seemed consistent, and was unremittingly negative for the Lib Dems.

This ICM poll is obviously different (though it is normally one of the more favourable ones for the Lib Dems, and has shown us at or above 21% five times this year). Only time will tell whether it’s an outlier or a herald of a different trend.

The next indication will come quite soon. I gather a MORI poll is due in the next day or two, together with a YouGov survey of marginals (though I don’t know whether it will include Lib Dem marginals).

by Clegg's Candid Friend on October 20, 2008 at 11:39 pm. Reply #

I’m not sure the variance is as much as people claim. Yes, these last two polls include both the bottom end and the top end of recent LibDem polling numbers.

If you assume we’re at about 17%, then virtually all recent polls fit within the margin of error. The Tories seem to be at about 42% and Labour at about 29%.

by Mark Littlewood on October 20, 2008 at 11:48 pm. Reply #

No, I think there is a clear difference between this poll and the others. This ICM poll shows us above our average rating for the year so far.

The most recent polls from the other 5 polling organisations all show us at or below our low for the year so far.

Rather than just splitting the difference it would make more sense to attach greater weight to the 5 that agree. But only time will tell who is right – ICM or all the others.

by Clegg's Candid Friend on October 21, 2008 at 12:19 am. Reply #

It would be hard to believe anything has happened in the last few days to set a new trend in our poll rating – apart from it possibly taking a small knock owing to struggle to get much coverage in the past week or do.

The margin of error is 3% or so, these sort of variations just don’t mean very much at all.

by Mark Littlewood on October 21, 2008 at 12:25 am. Reply #

It’s a weird poll…not least because our 4% gain comes from the ‘others’ section which doesnt seem very plausible to me…

by Darrell on October 21, 2008 at 12:44 am. Reply #

Hasn’t there just been a sudden rise in your ratings to over 20%?

by asquith on October 21, 2008 at 7:20 am. Reply #

Others – SNP – could Salmond’s govt be coming off the boil and its voters turning to the Lib Dems? Tis plausible.

by Thomas Hemsley on October 21, 2008 at 7:28 am. Reply #

Thomas Hemsley – ICM do include a don’t know option, everybody does. The difference is what they do with those don’t knows.

YouGov, MORI and ComRes all dismiss don’t knows – making the assumption that would either be unlikely to vote, or that their votes would split in roughly the same proportion as other people (or that it isn’t a pollsters place to second guess what people will do).

ICM and Populus both reallocate a proportion of the don’t knows on the assumption that they will vote in the same way as they say they did in 2005. Populus reallocates 50% of former Conservative and Labour voters, and 30% of former Lib Dem voters (which seems unfair, but is on the back of how interviewees behaved in Populus’s 2005 polling). ICM reallocate 50% of former Con, Lab and LDem voters.

Phew. In short, ICM’s treatment of don’t knows is the most favourable to the Lib Dems (at the moment – at various times in the past the reallocation has helped the Tories and Labour). Not that it is all down to that, there will be sampling and weighting differences too.

by Anthony on October 21, 2008 at 12:29 pm. Reply #

Ipsos/MORI monthly poll:
CON 45%(-7)
LAB 30%(+6)
LDEM 14%(+2)

Changes are since last month – though that was the weird one that had the Tories on 52% and the Lib Dems on 12%.

So not really any confirmation here of the jump in Lib Dem support shown by ICM.

by Clegg's Candid Friend on October 22, 2008 at 1:53 pm. Reply #

The polls are behaving perversely at the moment, hein?

I say ignore them and let’s go canvassing ourselves!

by Oranjepan on October 22, 2008 at 1:55 pm. Reply #

YouGov poll for the Sunday Times in Scotland:

CON 20%(+3)
LAB 38%(+6)
LDEM 11%(-2)
SNP 29%(-5)

Holyrood (constituency):
CON 14%(+1)
LAB 31%(+5)
LDEM 12%(-3)
SNP 39%(-3)

Holyrood (regional):
CON 16%(+2)
LAB 29%(+4)
LDEM 11%(-3)
SNP 32%(-3)

Changes are since last month. For reference, the changes in Westminster voting intention since the 2005 general election are:
CON 20%(+4)
LAB 38%(-2)
LDEM 11%(-12)
SNP 29%(+12)

by Clegg's Candid Friend on October 26, 2008 at 11:33 am. Reply #

ComRes poll for the Independent:
CON 39%(-1)
LAB 31%(-)
LDEM 16%(-)

On these figures, online seat projection algorithms indicate a hung parliament. And that’s assuming more than 30 Lib Dem seats will be lost to the Tories. If that were scaled down to (say) 20 or so, for reasons discussed before, it would leave the Tories well short of a majority.

by Clegg's Candid Friend on October 28, 2008 at 10:19 am. Reply #

Anthony Wells on UK Polling Report discusses a different type of poll carried out by ComRes for an organisation called Theos.

Among the findings was that 20% would not vote for a leader who was an atheist (other categories: 72-year-old 43%; gay/lesbian 23%; Muslim 23%; woman 7%; divorced person 7%; Christian 7%; black 5%).

I’m tempted to say we should go for an octogenarian black lesbian atheist divorcee (the daughter of a Muslim/Christian mixed marriage) next time, just to spite the electorate. But sadly I don’t think we have an MP who fits that description, following the narrow defeat of Fatima O’Reilly in Tunbridge Wells.

But seriously, I’m surprised that an atheist leader would be almost as unpopular as a homosexual one, and I wonder if some of our opponents may be tempted to make capital out of Clegg’s pronouncements on the issue.

by Clegg's Candid Friend on October 28, 2008 at 3:39 pm. Reply #

“I’m tempted to say we should go for an octogenarian black lesbian atheist divorcee (the daughter of a Muslim/Christian mixed marriage) ”

Typical! He always picks family… [/not the nine o’clock news]

by Jennie on October 28, 2008 at 3:44 pm. Reply #

YouGov poll for the Daily Telegraph:
CON 42%(-)
LAB 33%(-1)
LDEM 15%(+1)

Changes are since the last YouGov poll about 10 days ago.

by Clegg's Candid Friend on October 30, 2008 at 10:40 pm. Reply #

Leave your comment


Required. Not published.

If you have one.