Sunday, November 14, 2004

Over at the Volokh Conspiracy, Jim Lindgren quotes at length from a column purporting to debunk election conspiracy myths:
CLAIM: There was a suspicious difference between the exit polls and the final results.

My goodness! All of a sudden the art of polling, which my Democratic friends were insisting was unreliable right up until the election, is now is to be taken as gospel. Exit polls are "never wrong."

The exaggerations continue to grow. Kerry's lead in the exit polls keeps getting bigger. The polls' margin of error keeps getting smaller.

The Florida exit polls from Election Day are lined up on my desk. The biggest lead Kerry had was 51-49. The last update showed 50-50. The actual result was 52-47. Within the margin of error. Sorry.
First, lets note that the accusation of hypocrisy is rather ridiculous: before the election, many democrats alleged that telephone polling did not reach the young,hip crowd with only cell phones; further they argued that polls will underestimate Kerry's support at the polling booth since undecideds will split for the challenger, as they historically have done. Whether these objections were true or not, its fairly obvious that they are not applicable to exit polls - so its difficult to see where the contradiction lies in voicing these objections before the election and trusting exit polls now.

As for the "debunking" offered - it rather misses the point. Out of the sixteen states for which exit poll estimates are available, Kerry underperformed his numbers 15 out of 16 times. Almost every time, the result was within the margin of error (and in NH, Kerry's drop was actually outside the MoE). Now if exit polls were an unbiased estimate of the election outcome, we'd expect each candidate to outperform his numbers roughly half the time. Toss 16 coins and see how many times you get at least 15 heads (the answer is 2.6 in 10,000, in case you're curious).

In short, Bush's overperfomance of the exit polls was an extremely unlikely event that is difficult for me to explain.

So does this mean Bush stole the election? Not quite.

For one thing, its possible that something went wrong with the exit polls. There has been speculation that Republicans refused to participate more often than Democrats. While there is no actual evidence for this, its certainly a possibility, and generally there is no shortage of things that could go wrong in a poll.

Further, a problem is that we have no idea about the quality of the data we are relying on. The released data has been weighed to match the election returns. What about the raw data, not weighed to give the counted Bush/Kerry percentages? Only the consortium of TV Networks & newspapers which commissioned the exit poll has access to it. How about releasing the raw data to the public? While we toss around numbers - given to us by Slate on election day and various other media-friendly bloggers - but we have no idea how reliable these numbers are and we don't know what stage of the weighing process they are from - are they unweighed to match election returns, partially weighted, weighed at the precint level, weighted at the state level, etc. Until the raw data is released, its difficult to analyze the differences between the exit polls and the outcome.


Post a Comment

<< Home