Friday, August 28, 2015

i'm glad somebody did this, but it's youse guys...

first off, ivr actually has a far better track record than self-selecting online panels. this idea that you've got kids mashing numbers or whatever is kind of silly. but, the lack of randomness in the online polling is a significant problem - and this kind of polling has tended to do very poorly in shifting elections near election dates as a result of it [the last ontario election is an example where phone polling was very accurate and online polling was way off]. in stable campaigns, it's less of an issue. in this election, i would expect to see wide fluctuations, which is the particular weakness of the panel method.

after the bc election, firms have indicated that they will pay closer attention to overweighting the youth vote. they claim they got it wrong because they weighted young people by their size in the census, rather than their likelihood to vote. that correction would have taken most of the polling into the margins. this is a valid thing to point out, but it's not dependent on the survey method - and they claim they're adjusting for it. we'll see how that works out.

that said, this particular ivr poll is probably pushing the margins a little. that's an important point you forgot to mention. when a polling firm says they've got the ndp at 40 and the conservatives at 23 with a 3% margin of error, 19 times out of 20, what that actually means is that they claim there's a 95% chance that the ndp are between 37 and 43 and the conservatives are between 20 and 26. a little common sense will suggest that the ndp are probably closer to 37, and the conservatives are probably closer to 26. note that that also puts the liberals between 27 and 33, and comparisons to previous polls would suggest they're probably still in the 20s, anyways.


but, expanding on the regional part - i think the ivr polling had the ndp at 32% in alberta. that's a pretty good showing in a federal election for a party that isn't the conservatives. and, given the circumstances, it's entirely believable. but, it's really just good for second place everywhere outside of edmonton. the difference between 34 and 37 nationally would not be much if it works out purely to that kind of boost in alberta. they also had them in first in ontario, which is less believable. but, even a nice ndp bump in ontario is probably not going to make a lot of difference if it's almost entirely in traditional conservative-liberal races, and just ends up splitting the vote - while not making a dent in the conservative dominated rural areas. the reality is that the conservatives could easily win a plurality of 50+ seats in ontario with 25% of the vote, given the nature of the rural/urban split - and that they get very quickly diminishing returns as that number increases. they could get walloped in toronto, and still end up with the most seats.

the polling also broadly indicates that support for the ndp is largely "leaning", and that current snap polls may consequently be exaggerating support for the ndp and underestimating support for both the bloc and the liberals.

something else to keep in mind is to look at the swings and ask yourself "does this make sense?". if you saw a poll with a 10% swing between the ndp and the conservatives, you'd be wise to be wary. in the ivr poll, there was an apparent 6% swing from the conservatives to the ndp, which had the firm claim it was clear that the ndp was gaining at the expense of the conservatives. but, you can disentangle that a little if you look at it carefully.

you could first take note of the margin of error to cut it down to 3%. and, then you might want to take note that the liberals went up by 2% and the greens and bloc each went down by 1%. that leaves a 1% swing between the conservatives and the ndp, which is likely accurate if it's restricted to alberta - which makes sense, in context.