Google News gets gamed by a crappy content farm

On a search about Dr. Laura, Associated Content comes up first. Is the age of the bot coming to an end?

Published August 20, 2010 3:21PM (EDT)

I was on vacation for much of the last couple of weeks, so I missed a lot -- including the self-immolation of Dr. Laura Schlessinger. Apparently Schlessinger was the last public figure in the U.S. who does not understand the simple rules of courtesy around racial/religious/ethnic slurs. (As an outsider you don't get a free pass to use them -- no matter how many times you hear them uttered by their targets.) She browbeat a caller with a self-righteous barrage of the "N-word" -- and wrote her talk-show-host epitaph.

I shed no tears for Dr. Laura -- why do we give so much airtime to browbeaters, anyway? -- and I don't care much about this story. But after reading a post over at TPM about Sarah Palin's hilariously syntax-challenged tweets defending Schlessinger, I wanted to learn just a bit more about what had happened. So of course I turned to Google.

Now, it may have been my choice of search term, or it may have been that the event is already more than a week old, but I was amazed to see, at the top of the Google News results, a story from Associated Content. A.C., of course, is the "content farm" recently acquired by Yahoo; it pays writers a pittance to crank out brief items that are -- as I've written -- crafted not to beguile human readers but to charm Google's algorithm.

A.C.'s appearance in the Google lead position surprised me. I'd always assumed that, inundated by content-farm-grown dross, Google would figure out how to keep the quality stuff at the top of its index. And this wasn't Google's general search index recommending A.C., but the more rarefied Google News -- which prides itself on maintaining a fairly narrow set of sources, qualified by some level of editorial scrutiny.

Gee, maybe Associated Content is getting better, I thought. Maybe it's producing some decent stuff. Then I clicked through and began reading:

The Dr. Laura n-word backlash made her quit her radio show. It seems the Dr. Laura n-word controversy has made her pay the price, as the consequences of herbrought down her long-running program. But even if it ended her show, it may not end her career. Despite being labeled as a racist, and despite allegedly being tired of radio, the embattled doctor still seems set to fight on after she leaves. In fact, the Dr. Laura n-word scandal has made her more defiant than ever, despite quitting.

I have cut-and-pasted this quote to preserve all its multi-layered infelicities. The piece goes on in this vein, cobbled together with no care beyond an effortful -- and, I guess, successful -- determination to catch Google's eye by repeating the phrase "Dr. Laura n-word" as many times as possible.

The tech press endlessly diverts itself with commentary about Google's standing vis-à-vis Facebook, Google's stock price, Google's legal predicament vis-à-vis Oracle, and so forth -- standard corporate who's-up-who's-down stuff. But this is different; this is consequential for all of us.

I was a fairly early endorser of Google back in 1998, when the company was a wee babe of a start-up. Larry Page impatiently explained to me how PageRank worked, and I sang its deserved praises in my Salon column. For over a decade Google built its glittering empire on this simple reliability: It would always return the best links. You could count on it. You could even click on "I'm feeling lucky."

I still feel lucky to be able to use Google a zillion times a day, and no, Bing is not much use as an alternative (Microsoft's search engine kindly recommends two Associated Content stories in the first three results!). But when Google tells me that this drivel is the most relevant result, I can't help thinking, the game's up. The Wagner tubas are tuning up for Googledammerung: It's the twilight of the bots.

As for Associated Content, it argues -- as does its competition, like the IPO-bound Demand Media -- that its articles are edited and its writers are paid and therefore its pages should be viewed as more professional than your average run-of-the-mill blogger-in-pajamas. I think they've got it backward. I'll take Pajama Boy or Girl any day. Whatever their limitations, they are usually writing out of some passion. They say something because it matters to them -- not because some formula told them that in order to top the index heap, they must jab hot search phrases into their prose until it becomes a bloody pulp.

Let me quote longtime digital-culture observer Mark Dery, from his scorcher of a farewell to the late True/Slant:

The mark of a real writer is that she cares deeply about literary joinery, about keeping the lines of her prose plumb. That's what makes writers writers: to them, prose isn't just some Platonic vessel for serving up content; they care about words.

The best bloggers know a thing or two about this "literary joinery." And even bad bloggers "care about words." But the writer of Associated Content's Dr. Laura post is bypassing such unprofitable concerns. He chooses his words to please neither himself nor his readers. They're strictly for Google's algorithm. The algorithm is supposed to be able to see through this sort of manipulation, to spit out the worthless gruel so it can serve its human users something more savory. But it looks like the algorithm has lost its sense of taste.

[I should state for the record that in the course of my business work for Salon.com I had occasion to meet with folks from Associated Content. They were upright and sharp and understood things about the Web that we didn't, then. They've built a successful business out of "content" seasoned to suit the Googlebot's appetite. It's just not what we think of when we think of "writing." And if this piece is any indication, there isn't an editor in sight.]

BONUS LINK: If you want to understand more fully the process by which "news" publishers watch Google for trending topics and then crank out crud to catch Google's eye, you cannot do better than this post by Danny Sullivan of SearchEngineLand. Sullivan calls it "The Google Sewage Factory":

The pollution within Google News is ridiculous. This is Google, where we're supposed to have the gold standard of search quality. Instead, we get "news" sites that have been admitted -- after meeting specific editorial criteria -- just jumping on the Google Trends bandwagon...


By Scott Rosenberg

Salon co-founder Scott Rosenberg is director of MediaBugs.org. He is the author of "Say Everything" and Dreaming in Code and blogs at Wordyard.com.

MORE FROM Scott Rosenberg


Related Topics ------------------------------------------

Google Media Criticism