In early October, the Gallup polling company told Politico that it would not be doing horse race polling in the current presidential primary race, and perhaps not in the general election, either. “We believe to put our time and money and brainpower into understanding the issues and priorities is where we can most have an impact,” Gallup editor-in-chief Frank Newport said.
It came as a surprise, if not a shock. In 2012 Gallup had predicted a narrow Mitt Romney win, instead of the comfortable Barack Obama victory that was actually achieved, but a serious post-mortem seemingly ensured it would continue its long tradition, dating back to 1936, of polling presidential races.
Less than a month later, on Nov. 4, the Louisville Courier-Journal and Lexington Herald-Leader announced major changes after their pollster, SurveyUSA, missed Matt Bevin's surprisingly strong win in the Kentucky governor's race. SUSA was hardly alone in getting the outcome wrong, however — as shown by the polling trend averages at both Huffington Post and Real Clear Politics. But it did not help that the poll also found Mitch McConnell in a much closer race for his Senate seat in 2014, and he rolled to re-election over Alison Lundergan Grimes.
“It feels like déjà vu all over again,” Harry Enten wrote at Five Thirty Eight, linking back to a Nate Silver live update as last May's British election results showed pre-election polls had been far off. “The World May Have A Polling Problem," Silver's update was titled, pointing to the Scottish independence vote, the Israeli legislative election, David Cameron's relative landslide and last year's U.S. midterms. “It’s become harder to find an election in which the polls did all that well,” Silver wrote at the time.
As recently as August 2014, Silver had written about the problems facing pollsters, primarily driven by falling response rates (down from 36 percent in 1997 to 9 percent today, according to Pew Research), but still could say, “We have seen no widespread decline in the accuracy of election polls, at least not yet.” Fifteen months later, that no longer seems to be true.
Even when Silver wrote, there was Gallup's black eye to consider, but Gallup had been an outlier. Most pollsters correctly predicted Obama's victory, though they did under-estimate it overall. Now, the difficulties have become more widespread, as if the problems driven by falling response rates have finally reached a tipping point. In the case of Kentucky, Huffington Pollster's weekly roundup pointed to a number of factors:
Although it’s difficult to know for sure, evidence points to the timing of the polls, the partisan composition of likely voters in the polls and the presence of an independent candidate. None of these alone explains the wide discrepancy between the polls and the result, but combined, they contributed to the miss.
It went on to add the sparsity of polling as well: “only six publicly released polls between August and Election Day.”
The proliferation of these problems points to an even broader phenomena than the falling response rate — an ongoing dissolution of political trust, coherence and order. Low-turnout elections in and of themselves reflect such a general democratic decay, while third-party candidates tend to signal systemic problem-solving failures and voter disaffection. Pollster Charles Franklin also highlighted how pollsters matched party registration, but not identification — a divergence that further underscores the process of dissolution.
Over on center stage, in the GOP presidential primary, there are a multitude of other ways in which this dissolution plays out, starting with the deep distrust of the political establishment reflected in the success of Donald Trump and Ben Carson, but also extending to the difficulties of polling — much less covering — such an enormous primary field. The RNC's decision to let media hosts set the criteria, coupled with media decisions to use national poll aggregates and a two-tier debate format, actually empowers some of the worst fragmenting potentials that public polling can have.
But what about their potential for good?
Polling And Deliberation
Back in 2004, Gallup's Newport published a strong defense of what Gallup then did, “The Pluses of Pre-Election Horse Race Polls.” In it, he took note of Stanford professor James Fishkin and his advocacy of deliberative polling, centered on “deliberative forums in which citizens meet and talk in a modern-day town-hall setting, based on his belief that citizens involved in such discussions might arrive at better, more informed decisions” — only to quickly dismiss him.
“But just how much does this in-person deliberating differ from the broader input from friends and neighbors provided by polls? I think not much,” Newport wrote. “Polls, in fact, are in many ways an extension of a town-hall, deliberative process. Polling simply distills the opinions of all of one’s neighbors (across the state or across the nation) in an accurate fashion — an expanded version of what the voter would find out by asking neighbors in the community where they stand on the issues of the day.”
This carelessly overlooks the very essence of the deliberative process — considering different alternatives in light of different information, and discussing the various pluses and minuses that emerge. Nothing remotely like this happens in the course of most horse race polling. But Gallup's more recent decision to refocus on “understanding the issues and priorities” could potentially signal a move in that direction, a strategy for responding to the dissolution described above by finding another footing for the deliberative process. It's clearly a reactive response to the most obvious symptoms — problems with predicting the horse race outcome — but if Gallup is really serious, then it needs to go much farther, and confront some deeper problems.
For one thing, issues and priorities are not at all similar for all potential voters. As I noted here in late September, Philip Converse presented evidence in 1964 that “Those with low political knowledge have little to organize their political views. They respond mostly to the ephemeral nature of the times. Then, as voters become more knowledgeable, groups become very salient, to be replaced by ideological thinking among only the most informed.”
This is not to say that public opinion is meaningless, or that public deliberation is impossible — but it is tricky. You can't just assume that everyone responds to political questions in similar ways, which can produce coherent deliberation. They don't. This is also a significant problem with with Fishkin's approach. As described on his web site, the process begins with polling a random representative sample of the public, just like a normal poll, but then moves into a phase where participants are recruited for an in-person deliberative process. This process is extremely unlikely to engage the sorts of low-information citizens Converse identified as making up a substantial portion of the electorate. So how truly representative can it actually be?
The Public Interest Polling Model
Fortunately, another pollster has taken important steps in the direction of showing how deliberation can be made possible, without asking anyone to do much more than an ordinary poll would. In 1986, shortly before Fishkin began his work, Alan F. Kay launched his model of "public interest polling." The Americans Talk Issues Foundation's series of more than 30 polls continued over the next decade and a half.
As Kay explained the concept in 1999, “Public-interest polling is about issues — resolving community, regional, state, national and international problems. What is needed is governance that will help make the world, or our part of it, work with consensus support as easily realizable as a small town-meeting can find a consensus should it wish to.” Although it doesn't engage participants in back-and-forth discussions with one another, it does include methods for exposing people to different arguments and discovering if they significantly change people's views. And it doesn't require any higher levels of political sophistication or commitment. Like the jury system, it's open to one and all.
“Public-interest polling requires a healthy mixture of common sense and the knowledge of experts,” Kay wrote. “Questions must be fair,” they should not "lead the witness," as lawyers would say. “Question-sets (not necessarily every question) should be balanced. The survey design team, consisting of polling and issue experts, should collectively represent a wide range of backgrounds and points of view.” And they should be willing to work through any confusions that arise, using “follow-on questions,” or split samples to pose slightly different versions for comparison, as well as serial surveys.
“Once you’ve gotten the hang of it, you can clear up all the ambiguities in one or two additional surveys,” Kay writes. “In 33 surveys, over and over again, it worked every time for my company, ATI, from 1987 to 1999.” Kay's findings are collected in his book, Locating Consensus for Democracy - A Ten-Year U.S. Experiment.
Perhaps what's most striking about Kay's approach is the way he has surfaced consensus positions on a wide range of issues that most pollsters never even ask about and that few political elites ever consider. The unexpected consensus revealed should not be surprising, given the findings reported by Martin Gilens and Benjamin Page about the powerlessness of the general public. Based on a data set covering 1,779 policy issues, they reported:
Multivariate analysis indicates that economic elites and organized groups representing business interests have substantial independent impacts on U.S. government policy, while average citizens and mass-based interest groups have little or no independent influence.
In this situation, all manner of popular policies are simply ignored, unless they happen to coincide with what some powerful interest group also wants.
Kay's results also call to mind the “Big Ideas” poll commissioned by the Progressive Change Institute in January, which I wrote about last summer. “PCI first solicited ideas online through an open submission process (more than 2,600 specific proposals were submitted) and then let people vote on them (more than a million votes were cast),” I explained. “This bottom-up process was then tested out in a national poll.”
It was a very different sort of process than Kay's but, like his approach, it tapped into neglected sources of strong public consensus. Ideas receiving 70 percent support or more included:
Allow Government to Negotiate Drug Prices (79 percent)
Give Students the Same Low Interest Rates as Big Banks (78 percent)
Universal Pre-Kindergarten (77 percent)
Fair Trade that Protect Workers, the Environment, and Jobs (75 percent)
End Tax Loopholes for Corporations that Ship Jobs Overseas (74 percent)
End Gerrymandering (73 percent)
Let Homeowners Pay Down Mortgage With 401k (72 percent)
Debt-Free College at All Public Universities (Message A) (71 percent)
Infrastructure Jobs Program — $400 Billion / Year (71 percent)
Require NSA to Get Warrants (71 percent)
Disclose Corporate Spending on Politics/Lobbying (71 percent)
Medicare Buy-In for All (71 percent)
Close Offshore Corporate Tax Loopholes (70 percent)
Green New Deal — Millions Of Clean-Energy Jobs (70percent)
Full Employment Act (70 percent)
Expand Social Security Benefits (70 percent)
However, these ideas were all developed from a self-selected progressive base, and were not subject to winnowing through exposure to counter-arguments. What's fascinating about Kay's approach is that it also produced robustly popular proposals that political elites rarely considered, and the greater diversity among them has important lessons for us today, several decades after they were first uncovered.
Kay also wrote that ATI found “overwhelming evidence that the legislation and policy choices most supported by Americans are characterized by four key adjectives: stable, consistent, pragmatic and principled.” They vary only slowly over time, except in response to some major event; levels of support vary only slightly when questions are presented in different ways; solutions are pragmatic, rather than ideological, often encompassing “other dimensions” that aren't addressed by different party proposals and “they want solutions that are good for everybody and comprehensive,” rather than simply doing what's best for themselves alone.
The full range of ATI's findings were published in Kay's book, Locating Consensus for Democracy - A Ten-Year U.S. Experiment, but a number of significant, noteworthy results are presented online as well. It's fascinating to consider five examples of consensus mass support for sound ideas that elites still haven't caught up with after 20 years, at least partly due to the power imbalance highlighted by Gilens and Page.
Examples of Surprising Consensus
The first concerns energy policy that's good for the economy and the environment. In 1991, ATI compiled 18 proposals, “covering the best ideas for expanding energy supplies from coal, oil, natural gas, nuclear, solar-based renewables, exotics and conservation/efficiency improvements of all kinds,' presenting them with the evaluative frame "Will this proposal get us the energy we need and help the economy and help the environment as well?" There were actually two questions used, (H1) asking if the proposal “will help us get the energy we need and will help our economic competitiveness and job creation” and (H2) asking if the proposal “will help us get the energy we need and help the environment as well.” There were five “triple winners” with majorities saying yes to both questions (except for one, which received 49 percent on one question). With the support levels on H1 and H2, they were:
- Install new renewable-energy electric generating systems based on wind, solar and hydro or water power. (76 percent, 71 percent)
- Build cars meeting new high fuel efficiency standards. (61 percent, 68 percent)
- Build new high fuel efficiency trains and planes. (60 percent, 49 percent)
- In homes and in commercial and industrial buildings, install new efficient systems for lighting, for pumping and mechanical processes and for air conditioning, refrigeration and heating. (59 percent, 60 percent)
- Adopt, on a national scale, some new fuel — like hydrogen or alcohol fuels — to begin to replace gasoline. (59 percent, 60 percent)
That was ATI's poll result in 1991! Imagine how much farther along we'd be today in fighting global warming, if the pubic voice, reflected in ATI's results, had been heeded by politicians at the time.
Instead, Kay wrote, “The Johnston-Wallop National Energy Strategy bill, S. 2166, passed the Senate by an overwhelming 94-4 vote in Feb. 1992, with provisions almost diametrically opposed to the public’s preferences ... the Senate bill favored energy proposals that the public had turned down: build new safer nuclear power stations, allow oil development in coastal regions off-shore previously banned and streamline the approval process for new nuclear power plants.”
A second topic covered was environmental protection (cleaning up toxic waste) in an international framework. People are usually presented with something like the following three options:
- The government requires the clean-up at the taxpayer’s expense so that the industry does not move out of the country.
- The government compromises and requires only as much clean-up as it determines the industry can afford.
- The government requires the clean-up and takes the risk that our economy will suffer if the industry moves out of the country.
A pretest showed plurality support for B, but they decided to add another option:
4. The U.S. or the United Nations takes the lead in negotiating an agreement that would require all countries to enforce the same toxic waste standards for this industry.
In response, a two-thirds super-majority switched to the fourth option:
1: 5 percent; 2: 17 percent; 3: 9 percent; 4: 66 percent.
“Clearly here was an example of the public being ahead of both the leaders and the news media both at the time and to this day,” Kay wrote, in 1998. “In 1992, these results had the makings of a consensus position on how to handle recalcitrant polluting industries.” Almost 20 years later, the politicians have yet to catch up.
The third example concerns the role of the UN in global security. The question was asked three times in the 1990s — twice in 1991, and again in 1995. The initial question was "When faced with future problems involving aggression, who should take the lead, the U.S. or the UN?" The follow-up was, "If the UN refused to take the lead and a dictator was pursuing aggression against another country, what should the U.S. do: take the lead, wait for other nations to act, or stay out of it?" Here are the four options, with the levels of support in March 1991, June 1991 and June 1995:
- The UN should take the lead. (85 percent; 80 percent; 69 percent)
- The U.S. should take the lead, either immediately or if the UN fails to act. (n.a.; 62 percent; 54 percent)
- The U.S. should take the lead. (11 percent; 17 percent; 28 percent)
- The UN should take the lead and the U.S. stay out of it, if the UN does not act. (n.a.; 13 percent; 20 percent)
The super-majority consensus choice also happens to be international law, enshrined in the UN Charter, which the U.S. not only signed, but played a major role in shaping. Yet, our political elites routinely ignore both international law, and the collective wisdom of the American people. If the world today were largely peaceful, and free from threats, we might conclude that elites were right to ignore both the law and the people. But obviously that's not the case.
That's not the only place where the public supports a leadership role for the UN. A fourth example from ATI's work involves giving up U.S. sovereignty in favor of UN resolutions to protect the environment. Here's the proposal people were asked to approve or disapprove of:
In order to protect and preserve the world’s environment, United Nations resolutions on polluting the atmosphere and dumping toxic wastes in the ocean should have the FORCE OF LAW and rule over the actions and laws of individual countries with weaker environmental laws, even the laws of the United States when our environmental protection laws are weaker.
And the results:
Favor: 72 percent (37 percent “strongly”)
Oppose: 26 percent (12 percent “strongly”)
A more detailed account is given here, but it's clear that the vast majority of Americans are quite willing to embrace international law to deal with the environment. The virulent hatred and fear of the UN cultivated in the right-wing base simply doesn't resonate with the vast majority of the American people.
Consensus On Fixing Government
The fifth example from ATI is a much more detailed one. It's an exploration of developing consensus around government reform, a concern given great prominence by Ross Perot's candidacy in 1992. “The story begins with a well-known, long festering problem, the alienation and frustration of Americans with government, but soon becomes more positive,” Kay writes. First, in defining the problem, ATI found high levels of distrust, belief that government wastes a lot of money and is run for the benefit of a few special interests. Then ATI turned to looking at 38 possible policy responses.
Many in the first group were described as well-known, politically difficult and punitive. Examples included reducing the pay and staff of Congress, introducing term limits, limiting congressional contributions from lobbyists and special interests and passing a balanced budget amendment. Cutting congressional pay was the most popular proposal (80 percent in favor), while two forms of the balanced budget amendment were the least popular (59 percent), but all reflected a similar mood.
"The depth and persistence of the desire of the American people to cut Congressional pay, perks and benefits is overwhelming and persistent," Kay noted. "Clearly, punitive proposals will be strongly resisted by Congress and extremely difficult to enact," he observed, leading naturally to the next question, "Can the deep public frustration be mollified without resorting to punitive extremes?"
This lead to the second, more pro-active group of reform proposals, which included all of the following with 70 percent support or more:
- Make voter registration easier by the motor-voter plan where you register to vote at the same time you get your driver's license.
- Include a voluntary questionnaire with IRS tax forms so that you can tell the government your priorities on the principal items in the federal budget.
- In the same way we've developed and use the Gross National Product to measure the growth of the economy, develop and use a scorecard of new indicators for holding politicians responsible for progress toward other national goals, like improving education, extending health care, preserving the environment and making the military meet today's needs.
- Require Congress to conduct scientific, non-partisan, large sample surveys of public opinion on all important national issues and to promptly release the results to the media so that Congress and the public will know how most Americans view these issues.
- Require Congress to conduct scientific, non-partisan, large sample surveys of public opinion on all important national issues and to promptly release the results to the media so that Congress and the public will know what most Americans want for legislation.
There was also more than 60 percent support for different free airtime provisions including reinstatement of the fairness doctrine. In contrast to the first group, Kay characterized many of these as being low-cost and non-punitive, as well as highly supported:
Many of the proposals in group two are positive, pro-active and capable of opening up the government to the people as a whole. Many are as strongly supported by the public as most of the highly supported but punitive cut, limit and reform proposals. They should be quite palatable to, even welcomed by, Congress, as truly improving the quality of democracy in America, strengthening the republic, costing little, not interfering with any legitimate government function and, most important, not curtailing useful, reasonable means, support, prerogatives or rewards for the good service to the country that most Members of Congress do, in fact, supply.
Unfortunately, only the motor-voter plan was adopted by Congress. All the other measures that could have been adopted to produce a more vibrant, responsive democracy and build public trust were ignored. In retrospect, it's hardly surprising that the punitive ideas from group one continue to circulate, while the public mood has grown increasingly negative, so that now less than one person in 10 even thinks it's worthwhile to talk to a pollster. Nor is it surprising that a couple of inexperienced political outsiders continue to dominate the GOP presidential race.
The early 1990s, when ATI did this poll, are now more than 20 years in the past. The state of public disillusionment has only deteriorated since then. Now that's gone so far that it's threatening elites themselves — the GOP establishment, for one, Gallup for another — perhaps the time is finally ripe for taking up some of the ideas that ATI advanced.
After all, as Kay wrote back then, “They should be quite palatable to, even welcomed by, Congress.” With an 11 percent approval rating, what does Congress have to lose? Likewise pollsters with a 9 percent response rate, what do they have left to lose? The public has suffered a very long time from elite neglect. But now, as elites start suffering, too, we just might be approaching a tipping point, where forces drawing us together can regain a foothold. Pollsters following ATI's example could help enormously in that regard. Their problems and ours are intimately interlinked. So, too, are the potential solutions.