The facts vs. David Brooks: Startling inaccuracies raise questions about his latest book
Factual discrepancies in the NYT columnist's new book raise some alarming questions about his research & methods
Topics: David Brooks, Research, Statistics, The New York Times, The Road to Character, News, Politics News
For at least the past four years David Brooks, the New York Times columnist, TV pundit, bestselling author and lecture-circuit thought leader, has been publicly talking and writing about humility. Central to his thesis is the idea that humility has waned among Americans in recent years, and he wants us to harken to an earlier, better time.
One of the key talking points (if not the key talking point) cited by Brooks in lectures, interviews, and in the opening chapter of his current bestseller, “The Road to Character,” is a particular set of statistics — one so resonant that in the wake of the book’s release this spring, it has been seized upon by a seemingly endless number of reviewers and talking heads. There’s just one problem: Nearly every detail in this passage – which Brooks has repeated relentlessly, and which the media has echoed, also relentlessly — is wrong.
My journey down the Brooks rabbit hole began around two years ago, when I was conducting research for my book “Invisibles: Celebrating the Unsung Heroes of the Workplace.” The book largely consists of long-form profiles of professionals who thrive behind the scenes, and a lot of time is spent positioning these people as counterpoints to our contemporary culture of attention. In setting a historical context, I argue that there has been a shift away from self-effacement toward one where gaining recognition seems to be prized above all else. It’s easy to see the dovetail with Brooks’ premise about a shift from humility.
So it was that I came across video of a lecture he had given at the 2011 Aspen Ideas Festival, where he stated:
In 1950 the Gallup Organization asked high school seniors “Are you a very important person?” And in 1950, 12 percent of high school seniors said yes. They asked the same question again in 2006; this time it wasn’t 12 percent, it was 80 percent.
This seemed like a great stat to consider for my book, so I set about doing what I always aim to do when I come across a secondary source: I checked the facts. Before segueing to my current career as a writer, I spent about six or seven years as a magazine fact-checker, mostly for Condé Nast magazines; seeking primary sources for facts is my default mode.
I began by emailing a media contact at Gallup. After a series of exchanges, I was relayed to another Gallup employee. Ultimately I communicated directly with three different people at Gallup, and they communicated with multiple other staff, including a group in Princeton who combed Gallup’s print archives. During the process, a Gallup employee and I also emailed Brooks, but neither of us got a response. (Granted, I didn’t expect one, given that I had emailed his New York Times address, which likely receives a torrent of messages daily. I assume Brooks has another, more private email address he actually uses.) In their last email on the matter, the representatives at Gallup stated their final verdict: They did not “have any record of the data” that Brooks cited. But they also suggested that I try contacting Brooks again, and that if he shared any new information they would reopen the inquiry.
It was frustrating that they wouldn’t definitively just say Gallup didn’t conduct the polls Brooks cited, but if a number of people on the staff couldn’t find the polls then I knew either Gallup had atrocious record keeping, or Brooks was wrong. I hated that this tiny window of doubt was open, but there didn’t seem to be a path to find out one way or the other, so I gave up and didn’t include the passage in my book.
Fast forward to April 2015, when I came across a review of “The Road to Character” in the New York Times Book Review. To my amazement, the same statistic from the Aspen talk, once again credited to Gallup, was paraphrased. And my obsession with the mysterious Gallup poll roared back to life.
It was one thing if Brooks made erroneous claims in a one-off lecture. It was something else entirely to put those same claims in a book. Now I had to get to the bottom of this. I wanted to fact-check the review to make sure the reviewer was accurately quoting from the book. I did a search on Google Books for part of the quote and found it. After that, I went to Barnes and Noble and looked up the passage in the actual book just to confirm it was there.
The passage from “The Road to Character” reads:
“In 1950, the Gallup Organization asked high school seniors if they considered themselves to be a very important person. At that point, 12 percent said yes. The same question was asked in 2005, and this time it wasn’t 12 percent who considered themselves very important, it was 80 percent.”
Over the course of my search I discovered other iterations of the passage as well: During a 2011 appearance on “Real Time With Bill Maher,” for example, Brooks tells the same exact story, except in this telling, the second study took place not in 2005 or 2006, but in 1998.
The NYT’s review of “The Road to Character” itself picked up the pattern of changing dates, when the reviewer noted that the passage in question was similar to one in an earlier Brooks book, “The Social Animal,” only it was written “with slightly different dates.”
The passage from “The Social Animal” reads:
“In 1950 a personality test asked teenagers if they considered themselves an important person. Twelve percent said yes. By the late 1980s, 80 percent said yes.”
Somehow, between the publication of “The Social Animal” in 2011 and the publication of “The Road to Character” in 2015, a study that originally occurred, by Brooks’ telling, in “the late 1980s” became one that occurred nearly 20 years later. (Amazingly, to the New York Times reviewer, the late 1980s and 2005 are only “slightly different dates.” And how was any difference in dates for the same citation, no matter how “slight,” not problematic to the Times reviewer?)
What began as a simple fact-check of a Gallup poll was devolving into a morass.
I contacted Brooks’ publicist at Random House, and found the email address for a student at Yale who was thanked in the acknowledgments of “The Road to Character” as his fact-checker. The Yalie never replied, but I did get a response from his publicist, who told me that she had to check with Brooks, and that she would get back to me. It took over a week, but she replied with an admission that the study was wrongly ascribed to Gallup, but that the “sociological trend is accurate.” She also provided a link to what she said was the actual study (“Changes in Adolescent Response Patterns on the MMPI/MMPI-A Across Four Decades,” an analysis by university scholars of various data sets), and said the passage would be corrected in future printings of the book.
When I clicked the link and read the abstract, the first thing I noticed was the paper itself was published in 2003. “The Road to Character” stated that the data set was from 2005; how could that be when the final published paper came out two years earlier? That paper listed two relevant sets of data: One set collected in 1948 and 1954, and another set from 1989. I emailed Brooks’ publicist again. Had he merged the 1948 and 1954 polls into one that he then listed as having occurred in “1950”? And what the hell was up with the 2005 date listed in “The Road to Character”?
I did not receive a reply.
I then contacted two of the authors of the paper, Cassandra Newsom, a professor of pediatrics, psychiatry, & psychology at Vanderbilt University, and Robert Archer, a professor of psychiatry at Eastern Virginia Medical School. Abstracts and studies in general can be a challenge for the layperson to read, as academic text is often as opaque as legalese. So they sent me the actual paper and I went through it over the phone with Newsom. Here’s where things really began to unravel.
Newsom explained that, indeed, the first data set was from samples taken in 1948 and 1954, not 1950, and the second data set was from 1989, not 2005. Also, the first data set was exclusive to Minnesota, where respondents were disproportionately white, while the second was a more heterogeneous national sample. The respondents were not high school seniors, as Brooks wrote: In the ’48 and ’54 set, respondents were ninth graders; the 1989 sample was composed of 14- to 16-year-olds. (The ’48 and ’54 data set was later sifted to match the 1989 set by including only ninth graders who were aged 14-16.) Lastly, in the 1989 set, 80 percent of boys answered “true” to the statement “I am an important person,” and 77 percent of girls said true. Yet Brooks cited 80 percent as the only figure. I asked Newsom if it was possible that some other study was done in 2005 with the same “I am an important person” question. She said there has not been another normative sample since 1989.
(I emailed Brooks’ publicist a third time, asking her to contact Brooks for clarification on the stream of additional apparent inaccuracies but never heard back.)
The thing I keep wondering is how did Brooks get nearly every detail of this passage wrong? He said Gallup did the polls, when they were actually done by academics. He merged a data set from 1948 and 1954 into 1950. He said the second data set was from 2005, when it was from 1989 (to me, the most damning and damaging inaccuracy). He said it was high school seniors, when it was ninth graders. And he said 80 percent answered true, when that was only so for boys. Can one accidentally get this many details wrong?
