Pollsters' 2016 postmortem: National election surveys were accurate — it was the state polls that got it wrong

Hillary Clinton's lead slipped following FBI Director James Comey's letter — but she soon recovered, pollsters say

Published May 8, 2017 9:16PM (EDT)

 (Getty/Kayana Szymczak)
(Getty/Kayana Szymczak)

Since Donald Trump's unexpected win on election night last year, the conventional wisdom about the campaign polling has been that the survey industry completely botched things. Trump himself has repeatedly fed that perception by frequently deriding unfavorable polls as "fake news," even though he had admitted in December of last year that he expected he'd lose to Democrat Hillary Clinton. Contrary to popular perception, however, a new study conducted by the American Association for Public Opinion Research concludes that the polling at the national level was not inaccurate.

According to the 104-page AAPOR analysis, the national public surveys in aggregate indicated that Clinton was going to win the popular vote by a 3 percent margin. She ended up winning it by about 2 percent. That 1-point spread was "among the most accurate in estimating the popular vote since 1936," according to the report.

While the national pollsters were generally accurate in predicting the final countrywide tally for each candidate, their counterparts at the state level were not nearly as accurate, however. The AAPOR analysis listed three major factors for this discrepancy. In several states, many of the voters decided on their candidate at the last minute. These people overwhelmingly broke for Trump against Clinton. In Wisconsin, for example, 13 percent of voters made their decision during the last week of the campaign. Those voters chose Trump by a 30-point margin according to the study. In Florida and Pennsylvania, late-deciding voters broke for Trump by 17 points.

As more and more people have moved away from landlines to cell phones, the survey industry has begun adjusting its techniques to account for this change. But another demographic factor also began emerging in 2016 which was not factored in by some of the state-level pollsters: People who had lower educational attainments were less likely to respond to random questions about their political preferences. This mattered because, as the AAPOR report details, "Voters with higher education levels were more likely to support Clinton."

The 2016 campaign was unusual in that regard because, in past presidential elections, people who had not obtained a bachelor's degree normally voted Democratic whereas college graduates voted Republican quite heavily, as can be seen in the chart reproduced from the study below:

democratic margins

The analysis also found that many people who voted for Trump did not reveal themselves as supporting his insurgent candidacy until after they had cast a ballot for him.  That may have been in part due to the generally negative media coverage Trump received during his campaign but also because a large number of people who voted for him were Democrats who may not have wanted to disclose that they were taking a chance by voting for a Republican. According to another study conducted by a Democratic-linked super PAC, 77 percent of people who voted for Trump after having voted for Barack Obama believed that the New York businessman would create economic policies that would favor the middle-class or all Americans equally.

The AAPOR report also argued that several electoral models which claimed that Clinton had a greater than 90 percent chance of winning ahead of the election may have actually hurt her chances. "However well-intentioned these predictions may have been, they helped crystalize [sic] the belief that Clinton was a shoo-in for president, with unknown consequences for turnout," the report reads. The analysis also criticized several of the models for providing "down to the decimal point" probability estimates for the election outcome. Such seeming confidence gave site readers the impression that the estimators had the ability to predict things with "far more precision than warranted."

As for FBI Director James Comey's announcement to Congress (which soon leaked out to the public) that he was examining emails which were found on the computer of the husband of Clinton's closest advisor, AAPOR's regression analysis showed Clinton's numbers did decline immediately after the news broke. However, Clinton recovered and resumed her prior trajectory shortly thereafter.

The report acknowledged, however, that it is unknown just what caused late-deciding voters to make the Trump choice.

The full report can be viewed below:


By Matthew Sheffield

Matthew Sheffield is a national correspondent for The Young Turks. He is also the host of the podcast "Theory of Change." You can follow him on Twitter.

MORE FROM Matthew Sheffield