FiveThirtyEight issues brutally honest correction: “We will do better”

A post in the data-driven blog made several mistakes in its analysis

Topics: FiveThirtyEight, Nate Silver, statistical analysis, data, Nigeria, kidnappings, Journalism,

FiveThirtyEight issues brutally honest correction: "We will do better"Nate Silver (Credit: ABC)

FiveThirtyEight, the much-criticized data-driven blog founded by statistician Nate Silver, has issued a brutally honest correction for a series of errors in a post that attempted to show the “rapid acceleration” of kidnappings in Nigeria, broken down by geographic location.

“This article contains many errors, some of them fundamental to the analysis,” begins the editor’s note for a post dated May 13th, “Mapping Kidnappings in Nigeria.” The errors include misinterpreting and misrepresenting base data and failing to normalize the data shown in two data visualizations. The full note, below:



This article contains many errors, some of them fundamental to the analysis.

The article repeatedly refers to the number and location of kidnappings. But the Global Database of Events, Language and Tone (GDELT) — the data source for the article — is a repository of media reports, not discrete events. As such, we should only have referred to “media reports of kidnappings,” not kidnappings.

This mistake led to other problems.

We should not have published an animated map showing “kidnappings” over time, or even “media reports of kidnappings” over time. Because we have no data on actual kidnappings, showing a time series requires normalizing the data to account for the increasing number of media reports overall. Thus, showing individual media reports is a mistake. The second map, showing “Kidnapping rate per 100,000 people, 1982-present,” has the same flaw.

The animated map also incorrectly locates some reported kidnappings. If the location of a reported kidnapping isn’t in a media report, GDELT defaults the location to the center of Nigeria. So that part of the country is overrepresented in the animated map.

The article also should have made clear that while GDELT makes an effort to remove duplicate media reports of the same event, it is not always successful in doing so because media reports often conflict with one another. There were many conflicting reports about the mass kidnapping of hundreds of girls from the Government Girls Secondary School in Chibok in April. This likely accounts for at least some of the rise in media reports of kidnappings referred to in the last paragraph of the post.

The error is a serious one not only because FiveThirtyEight’s sole function is data analysis, but also because ESPN’s young blog has yet to prove its worth to its many skeptics. However, it seems that FiveThirtyEight is taking this seriously, ending the note with, “This piece did not meet FiveThirtyEight’s standards for publication. We apologize for the mistakes. We will do better.”

Prachi Gupta

Prachi Gupta is an Assistant News Editor for Salon, focusing on pop culture. Follow her on Twitter at @prachigu or email her at pgupta@salon.com.

More Related Stories

Featured Slide Shows

  • Share on Twitter
  • Share on Facebook
  • 1 of 13
  • Close
  • Fullscreen
  • Thumbnails
    Clare Barboza/Bloomsbury

    Uncommon Apples

    Api Étoile

    Like little stars.

    Clare Barboza/Bloomsbury

    Uncommon Apples

    Calville Blanc

    World's best pie apple. Essential for Tarte Tatin. Has five prominent ribs.

    Clare Barboza/Bloomsbury

    Uncommon Apples

    Chenango Strawberry

    So pretty. So early. So ephemeral. Tastes like strawberry candy (slightly).

    Clare Barboza/Bloomsbury

    Uncommon Apples

    Chestnut Crab

    My personal fave. Ultra-crisp. Graham cracker flavor. Should be famous. Isn't.

    Clare Barboza/Bloomsbury

    Uncommon Apples

    D'Arcy Spice

    High flavored with notes of blood orange and allspice. Very rare.

    Clare Barboza/Bloomsbury

    Uncommon Apples

    Esopus Spitzenberg

    Jefferson's favorite. The best all-purpose American apple.

    Clare Barboza/Bloomsbury

    Uncommon Apples

    Granite Beauty

    New Hampshire's native son has a grizzled appearance and a strangely addictive curry flavor. Very, very rare.

    Clare Barboza/Bloomsbury

    Uncommon Apples

    Hewes Crab

    Makes the best hard cider in America. Soon to be famous.

    Clare Barboza/Bloomsbury

    Uncommon Apples

    Hidden Rose

    Freak seedling found in an Oregon field in the '60s has pink flesh and a fragrant strawberry snap. Makes a killer rose cider.

    Clare Barboza/Bloomsbury

    Uncommon Apples

    Knobbed Russet

    Freak city.

    Clare Barboza/Bloomsbury

    Uncommon Apples

    Newtown Pippin

    Ben Franklin's favorite. Queen Victoria's favorite. Only apple native to NYC.

    Clare Barboza/Bloomsbury

    Uncommon Apples

    Pitmaston Pineapple

    Really does taste like pineapple.

  • Recent Slide Shows

Comments

Loading Comments...