As a great philosopher once observed, “Math class is tough!”
This insight has been confirmed numerous times in the past few weeks, as various pundits have taken innumerate pot shots at Nate Silver, the New York Times blogger and author, who as of today estimates that President Obama has a 77.4 percent chance of winning reelection next week.
The Villager gossip site Politico has featured several criticisms from its stable of contributors, who in the great tradition of political journalism have not allowed their ignorance of a subject – in this case probability theory – to keep them from opining on it.
Behold the wit and wisdom of Josh Gerstein and Dylan Byers: Gerstein asks, “Isn’t the basic problem with the Nate Silver prediction in question, and the critique, that it puts a percentage on a one-off event?” Meanwhile, Byers concludes that, “should Mitt Romney win on Nov. 6, it’s difficult to see how people can continue to put faith in the predictions of someone who has never given that candidate anything higher than a 41 percent chance of winning.”
Clearly it’s time for some remedial stats classes. Let’s start with a quick quiz. Suppose a weather forecasting model predicts that the chance of rain in Chicago tomorrow is 75 percent. How do we determine if the model produces accurate assessments of probabilities? After all, the weather in Chicago tomorrow, just like next week’s presidential election, is a “one-off event,” and after the event the probability that it rained will be either 100 percent or 0 percent. (Indeed, all events that feature any degree of uncertainty are one-off events – or to put it another way, if an event has no unique characteristics it also features no uncertainties.)
The answer is, the model’s accuracy can be assessed retrospectively over a statistically significant range of cases, by noting how accurate its probabilistic estimates are. If, for example, this particular weather forecasting model predicted a 75 percent chance of rain on 100 separate days over the previous decade, and it rained on 75 of those days, then we can estimate the model’s accuracy in this regard as 100 percent. This does not mean the model was “wrong” on those days when it didn’t rain, any more than it will mean Silver’s model is “wrong” if Romney were to win next week.
What Silver is predicting, in effect, is that as of today an election between a candidate with Obama’s level of support in the polls and one with Mitt Romney’s level of support in those polls would result in a victory for the former candidate in slightly more than three out of every four such elections.
This is a testable hypothesis, no different in kind from a weather forecast that predicts the chance of rain on particular days based on the available data (data that obviously become more reliable as the day in question approaches). Presidential elections are much less common than rainy days, but elections in general are taking place all the time, and Silver’s track record in regard to evaluating the probability of election outcomes is excellent.
For complicated psychological reasons, human beings tend to prefer their intuitions to rational statistical analysis. Consider this (in)famous problem, which, despite its very straightforward answer, even many mathematically sophisticated people get wrong:
Suppose you're on a game show and you're given the choice of three doors. Behind one door is a car; behind the others, goats. After you have chosen a door, the door remains closed for the time being. The game show host, Monty Hall, who knows what is behind the doors, now has to open one of the two remaining doors, and the door he opens must have a goat behind it. After Monty Hall opens a door with a goat, he will ask you to decide whether you want to stay with your first choice or to switch to the last remaining door. Imagine that you chose Door 1 and the host opens Door 3, which has a goat. He then asks you "Do you want to switch to Door No. 2?" Is it to your advantage to change your choice?
I estimate there’s an 87 percent probability that Joe Scarborough wouldn’t switch to Door No. 2.
Shares