SALON

Monday, Apr 22, 2013 3:28 PM UTC

How to prevent future Reinhart-Rogoff meltdowns

All economics papers can't be subject to a time-consuming peer review, but sweeping studies should raise a red flag

By

Topics: On the Economy, Reinhart-Rogoff, GDP, New York Times, Wall Street Journal, ,

How to prevent future Reinhart-Rogoff meltdownsU.S. Congressman Paul Ryan (R-WI) (Credit: Reuters/Shannon Stapleton)

In the midst of the Reinhart-Rogoff meltdown, a commenter was aghast to learn that their paper was not peer reviewed.* She asked, reasonably, how could the newspapers report findings that had not gone through that process?

It’s a fair question, and I should expand on the too glib remarks from my post:

So the answer is to only accept peer-reviewed work as economic knowledge, right?  Nope.  That would be a) too limiting, and b) wouldn’t advance the epistemological cause as much as you think.  Peers have their own sets of biases, particularly as gate keepers.

First, had R&R gone through the peer-review process, I’m fairly confident that a) the spreadsheet error would NOT have been found, but b) the paper would have been sent back to them for failing to provide even a cursory analysis of the possibility of reverse causality (slower growth leading to higher debt/GDP ratios vs. the R&R claim of the opposite).  Re “a,” peer reviewers do not routinely replicate findings, though they should when possible (more work these days is with proprietary data sets which cannot legally be shared).

But given “b”—this influential work would perhaps not have seen the light of day without significant revision—how is it that I’m not calling for more peer reviewing?

That’s where the “too limiting” part comes in.  First, a lot of what’s important in economics is fairly simple analysis of trends—descriptive data—without the behind-the-scenes number crunching of the type R&R did.  You learn a lot about austerity, for example, by simply plotting GDP growth rate of countries engaged in it, or wage inequality by observing the movements of different wage percentiles over time.  Looking back on my own posts, you’ll see employment plotted against employment, real wage trends, growth rates…none of which take hardly any data manipulation and none of which depend on choices that would concern a reviewer.

So to insist that everything gets peer reviewed, including the presentation of descriptive data published by reliable sources (e.g., BLS or BEA), would be to raise the bar unnecessarily high.  Of course, that’s not to say that descriptive data presenters can’t make mistakes, and we do.  So in the best of all possible worlds, it would be better if such work was checked by peers before it was publicized, even on blogs.  Which brings me to the next problem.

PR takes a long time (in my experience, six months to a year), and policy makers and reporters will seek results more quickly.  If newspapers had rules that they would only print peer-reviewed findings, there would be a long lag between, say, the release of an employment report and analysis of the results.

Then there are the gatekeepers’ biases, as mentioned above.  I’ve published a precious few peer-reviewed papers,** but those more active in that world tell that papers that challenge conventional wisdom have trouble getting very far, regardless of quality.  And since many aspiring economists are intimately familiar with the biases of the gatekeepers, they are hesitant to test those boundaries for fear of not making it into the important journals.

So, what’s the answer?  The media gatekeepers themselves have to be highly vigilant, especially when the results they’re publishing are not transparent.  R&R’s work, for example, involving many different countries over many years, with results put into seemingly arbitrary bins (debt/GDP<30%, 30%-60%, etc.) might be a flag that would lead a non-technical reporter to ask a lot of their sources what they thought before running with it.

In a sense, I’m suggesting more on-the-spot peer reviewing when results are complex, counter-intuitive, or, as in R&R’s case, directional causality is being invoked.  This risks the dreaded “he-said, she-said” which can be off-putting to readers, and it can easily be overused.   For example, there’s nothing complex, counter-intuitive about work showing, e.g., high unemployment associated with weak wage growth or higher poverty, so a non-PR’ed paper making such connections might invoke less scrutiny and a newspaper write-up of it would not obviously require alternative explanations.

But really, and unfortunately, at the end of the day, when it comes to economics reporting of results, you have to know who to trust.  The WSJ editorial page, for example, is often not trustworthy in their use and choice of findings.  The NYT’s, much more so.  (I should and will find some examples.)  It’s better with reporters (vs. editorial writers) but there too you’ll find folks with thumbs on scales.  In fact, one of the things I do here (and many other econ-bloggers do the same) is try to catch problems and elevate articles that get it right.

And then there’s the fact that facts don’t matter anywhere near as much as they should anyway these days, a much bigger problem that cannot be solved by any amount of peer reviewing.

*This confused some aficionados, because it appeared in the peer-reviewed journal American Economic Review, but in the “Papers and Proceedings” edition, which is not PRed.

**Like most in my field of think tank work, anything beyond a blog, especially with serious number crunching, is reviewed by as many outsiders as I can get to read it, but the difference is that they don’t have the final say on publication.

Jared Bernstein joined the Center on Budget and Policy Priorities in May 2011 as a Senior Fellow. From 2009 to 2011, Bernstein was the Chief Economist and Economic Adviser to Vice President Joe Biden. Follow his work via Twitter at @econjared and @centeronbudget. More Jared Bernstein.

Next Article

Related Stories

More Related Stories

Featured Slide Shows

The week in 10 pics

close X
  • Share on Twitter
  • Share on Facebook
  • Thumbnails
  • Fullscreen
  • 1 of 11
  • This photo. President Barack Obama has a laugh during the unveiling of the George W. Bush Presidential Center in Dallas, Tx., Thursday. Former first lady Barbara Bush, who candidly admitted this week we've had enough Bushes in the White House, is unamused.
    Reuters/Jason Reed

  • Rescue workers converge Wednesday in Savar, Bangladesh, where the collapse of a garment building killed more than 300. Factory owners had ignored police orders to vacate the work site the day before.
    AP/A.M. Ahad

  • Police gather Wednesday at the Massachusetts Institute of Technology to honor campus officer Sean Collier, who was allegedly killed in a shootout with the Boston Marathon bombing suspects last week.
    AP/Elise Amendola

  • Police tape closes the site of a car bomb that targeted the French embassy in Libya Tuesday. The explosion wounded two French guards and caused extensive damage to Tripoli's upscale al-Andalus neighborhood.
    AP/Abdul Majeed Forjani

  • Protestors rage outside the residence of Indian Prime Minister Manmohan Singh Sunday following the rape of a 5-year-old girl in New Delhi. The girl was allegedly kidnapped and tortured before being abandoned in a locked room for two days.
    AP/Manish Swarup

  • Clarksville, Mo., residents sit in a life boat Monday after a Mississippi River flooding, the 13th worst on record.
    AP/Jeff Roberson

  • Workers pause Wednesday for a memorial service at the site of the West, Tx., fertilizer plant explosion, which killed 14 people and left a crater more than 90 feet wide.
    AP/The San Antonio Express-News, Tom Reel

  • Aerial footage of the devastation following a 7.0 magnitude earthquake in China's Sichuan province last Saturday. At least 180 people were killed and as many as 11,000 injured in the quake.
    AP/Liu Yinghua

  • On Wednesday, Hazmat-suited federal authorities search a martial arts studio in Tupelo, Miss., once operated by Everett Dutschke, the newest lead in the increasingly twisty ricin case. Last week, President Barack Obama, Sen. Roger Wicker, R.-Miss., and a Mississippi judge were each sent letters laced with the deadly poison.
    AP/Rogelio V. Solis

  • The lighting of Freedom Hall at the George W. Bush Presidential Center Thursday is celebrated with (what else but) red, white and blue fireworks.
    AP/David J. Phillip

  • Recent Slide Shows

  • Share on Twitter
  • Share on Facebook
  • Thumbnails
  • Fullscreen
  • 1 of 11

Related Videos

Comments

 8 Comments

Comment Preview

Your name will appear as username

You may use these HTML tags and attributes: <a href=""> <b> <em> <strong> <i> <blockquote>

follow salon

Most Read

Popular on Reddit

links from salon.com

vote
Would we give up burgers to stop climate change? New study suggests if we reduce our intake of animal products by 25%, we can halt climate change
589 points590 points | 235 comments
vote
Facebook has been used to find ex-lovers, childhood BFFs or the one who got away. He used it to find the guy who, without his consent, made him his drug mule.
236 points237 points | 22 comments
vote
“This could be a career ender for Michele Bachmann”
141 points142 points | 55 comments
vote
Poll: 3 in 10 voters say “armed revolution might be necessary” - 29% say they believe they'll have to act "in order to protect our liberties" in the next few years
45 points46 points | 16 comments
vote
Capitalism, not feminism, killed home cooking–and fixing our broken food system is gonna take a lot more than some privileged women embracing “new domesticity”
32 points33 points | 19 comments

From Around the Web

Presented by Scribol
listener_new_rect The Listener: "The Shelter Cycle": Raised in a cult

Recommendations