i'm going to post a bit of a public service announcement about polling in the election, because i can. i never specialized in statistics, in fact i hated stats, but i've taken a bit more of an appreciation for the approach of measuring uncertainty since then. i am literate in this topic, and the media that most people get their information from is not. i also worked on the phone for years when i was in school, which is important experience if you can contextualize it properly.

the reputation of polling has taken a big hit over the last few years. how could the polls be so wrong? well, what you're referencing weren't polls to begin with. and the actual polls really weren't that wrong at all - so long as you understand what a margin of error is.

so, this is what we saw happen: the media consults an array of "internet panels" for research, these panels are nowhere close and then the media blames the inaccuracy on "low response rates" or "bias in the data". the entire narrative is completely ridiculous. rather, the actual correct conclusion is as obvious as can be: internet polling is not random, and because it is not random there is no way to determine it's accuracy. they don't even have margins of error. it's consequently not even accurate to suggest that the internet polling was wrong; you can't define what "right" or "wrong" means when you can't calculate the error. the actual accurate analysis is that what the media refers to as "internet polling" is not actually polling. you should not try and analyze it - it can't be analyzed. it's only utility is as propaganda. you should simply completely ignore it.

which brings up the other problem. consider the ontario election in 2014, which had the following results:

liberals: 38.65
conservatives: 31.25
democrats: 23.75
greens: 4.84

the result was a liberal majority. yet, the polls also seemed to suggest a minority. were they wrong?

well, the margin of error was generally around 3%. in order for them to be "wrong" with a 3% margin of error, they would have to have fallen outside the following ranges:

liberals: (35.65, 41.65)
conservatives: (28.25, 34.25)
ndp: (20.75, 26.75)
greens: (1.84, 7.84)

in fact, most of the telephone polls a few days ahead of the election did fall in these ranges (or in a larger range, with a larger margin of error). the online polling was all over the place and completely inaccurate, but if we ignore it like we should then we see the polls were almost spot on. a poll that had the liberals at 36 and the conservatives at 34 with a 3% moe might have seemed to hint at a minority, but it would not have been "wrong". rather, the widespread forecast of minority results were a consequence of the media reading the polling incorrectly. not bad polling. poor mathematical literacy in the media. so, don't blame the pollsters for doing it wrong. rather, blame the media for not understanding how to read the polls.

there's not a lot of good polling being done right now. i know. i've surveyed it. you've basically got two options if you want to understand what is going on.

the first is the nightly tracking by nanos, which is here:

http://www.nanosresearch.com/main.asp

on the one hand, he's doing everything right: sufficiently large sample sizes, telephone polling so it's actually random, proper weighting, etc. his results, nationally, will be accurate, within the margin of error. they always have been before. and, that 1/20 bit is a statistical caveat that never actually happens.

on the other hand, his polling is functionally useless because the election is going to turn on regional differences, and his sample sizes are too small. it's a 5.5% margin of error in ontario, which is far too large to tell us anything at all useful about what's happening there.

so, nanos is accurate nationally and useless regionally.

the forum polling is using phones, but it's always wrong and i don't know why. they just have a very bad track record at every level. they're obviously doing something wrong at a basic level. they're the only firm putting the ndp anywhere close to a majority and it's having a disproportionate effect on the coverage. but, there's little reason to think it's actually accurate, given that they're always wrong.

besides nanos, the other two you need to take seriously are ekos and mainstreet. phones. big sample sizes. but mainstreet does not do regular polling.

which whittles the list down to a single reputable polling firm: ekos.

http://www.ekospolitics.com/

unfortunately, the numbers are not as good as any of us would like them to be.

1. conservatives (29.9, 33.7)
2. ndp (27.7, 31.5)
3. liberals (25, 28.8)

but, the actual thing that matters is ontario - a province where the ndp do not appear to be competitive.

the media is continuing to tell us that the ndp are ahead. if nothing changes, that narrative will likely remain static. we will then ask how the polls were so wrong, when they end up tied for second.

the reality is that the legitimate, reliable, actual polling continues to place the conservatives at the front of a close pack and that a sober analysis of the results in front of us would suggest any outcome other than a conservative minority remains highly unlikely - unless the conservatives collapse in ontario.

the polling will not be wrong. the analysis is wrong.