American sports fans who don't pay much attention to soccer might not know it, but 2016 was the year of what was likely the greatest sports upset of all time: Leicester City overcoming 5000-to-1 odds to win the English Premier League title. It's the equivalent of a middling college basketball team winning the NBA championship. It was an event that caused Leicester City fans to lose their minds, fans of favorites like Manchester City to weep into their beers, and unaligned sports fans to marvel at the improbability and glory of it all.
But what no one did was get on social media and start yelling at oddsmakers for not knowing, at the beginning of the season, that this once-in-a-lifetime event was going to happen. On the contrary, the sports media writes glowing stories about the lucky sonsabitches who got drunk and took that bet, tossing a pound or two on LC for the championship, mostly on a lark, and walked away with fat pockets for it. (A $5 bet at the top of the season would result in a $25,000 jackpot, for those who are keeping up at home.)
Sadly for professional oddsmaker Nate Silver, he's working in politics these days and not sports, and therefore is speaking to an audience that seems a little less up on the difference between a long-shot and a no-shot.
Silver and his team at FiveThirtyEight gave Donald Trump 20-to-1 odds against winning the Republican nomination back in September. Within milliseconds of Trump securing the nomination earlier this month, the madding crowd descended on Silver on social media, demanding an apology and possibly his head for supposedly getting it wrong.
Except....there's no real reason to believe he got it wrong. He gave Trump 20-to-1 odds. That put Trump in the realm of unlikely but not impossible. Less Leicester City and more like rolling a 20 on a 20-sided-die. Which, as any D&D player will tell you, happens more than you'd think.
Still, when the mob is demanding its pound of flesh, sometimes you have to give them something to shut them up, and so it goes with Silver, who published a piece at FiveThirtyEight Wednesday morning, titled, "How I Acted Like A Pundit And Screwed Up On Donald Trump".
The move worked. Silver's critics are now falling all over each other on social media to praise him for his humility and to intone about how great it is that he's learned his lesson and how he's going to be better in the future etc etc etc.
It seems few of them actually read the piece, or they'd know it's a dry explanation of how odds work. Which, if you absorb the lesson, shows you why it's really not cool to be busting Silver's chops for this, because he never said Trump was impossible. He just said he wouldn't bet on him, which was the smart thing to say back in September.
Either way, Silver isn't really apologizing for getting it "wrong". Even though he says that his team made "a big mistake" in letting their subjective sense that Trump is too ridiculous of a candidate to really pull this one out, most of his piece is a defense of calling Trump a long-shot. Then, as now, all the prevailing evidence showed that insurgent candidates with so little institutional support face steep odds. (You can soon add Bernie Sanders to the pile, strengthening his argument.) He called Trump a long-shot, not a no-shot, and most of his supposed mea culpa is reminding his critics of what was always true.
The present doesn't change the past, and thinking otherwise is a fallacy in oddsmaking known as the hindsight bias.
Still, Silver gives some ground to his critics, noting that the precedent examples they looked to of insurgent candidates with little institutional support are few in number, giving them a poor dataset to work with.
This is true enough, but, in his eagerness to meet his critics halfway, Silver gives too much ground. After all, the dataset of such candidates included many people who had, unlike Donald Trump, held office before and therefore had things like an existing constituency and campaign experience, such as Howard Dean, Rudy Giuliani, and Newt Gingrich, which probably distorts Trump's odds upwards.
When you look at the other candidates in the dataset who match Trump in terms of experience — i.e., businessmen who never held office before and but have a con artist appeal — the number of examples that Silver had in his dataset was one: Herman Cain. You can now add Carly Fiorina to that pile. Not exactly the examples you look to when arguing that Trump had better odds than he seemed to have in September.
If there's any lesson here, it's that oddsmaking in politics is less exacting and predictive than in sports, something Silver readily admits throughout this piece. Political datasets are smaller and the system has a lot more chaos, making it harder to predict the future based on the past.
Which is why it's a shame that Silver, under all this pressure from outsiders, entertains the possibility that more weight should be given to polls, even months before the race, than to historical indicators like economic trends and party endorsements that, right up until Trump, were considered important predictors.
I personally hope he is very careful before making that leap. As Silver knows, better than most, the fact that Trump won doesn't actually mean he wasn't facing down 20-to-1 odds. While systems should be adjusted with new information, it's also important not to fall for hindsight bias. (Silver says as much, warning about the dangers that Trump's win might cause folks to "overcompensate" for perceived errors, leading them to overrate the odds that the next flavor-of-the-month loony bird GOP candidate pulls it out.)
While I'm no mathematician, I'd argue that 20-to-1 odds against Trump still seem about right to me. The man had a couple of lucky breaks that allowed him to hit that 5% shot: The San Bernardino attacks right after the Paris attacks, his strongest critic (Fiorina) making a fool out of herself on Planned Parenthood at a critical time, another strong opponent (Ted Cruz) backing him instead trying to take him out early, and having 17 opponents to begin with.
Imagine if a long-shot team had a series of games where their opponents kept getting injured or having a bad night, leading to improbable wins. Trump had the same string of good luck, but that doesn't mean that the odds should be recalculated to take into account unpredictable factors like that.
That all said, Silver's biggest problem was not that he acted too much like a pundit. On the contrary, the statistics-centered approach likely blinded them to a few unique factors of this race, such as the oversized field and the assist that Cruz was giving Trump. If he'd been a little more like a pundit, focused on the present instead of the past, he might have been able to adjust his models a little more to account for those factors.
But the response shouldn't be to dismiss either data journalism or punditry. As sports fans know, the two can be wed together pretty neatly to give audiences a sense of the big picture. The data guys can tell us that Leicester City has 5000-to-1 odds, but the pundits can tell us how they managed to pull it off anyway.
Same story with Donald Trump. He was a long shot, and the fact that he won doesn't change that. The job of pundits and historians is not to deny that, but to look at all the various things that had to happen for a man for such an improbable event to occur.
Shares