Big data’s next frontier: Crowd-testing fiction

Will the novels of the future be market-researched to please the public, just like Hollywood blockbusters?

Topics: Books, Fiction, Writers and Writing, Readers and Reading, Charlaine Harris, Publishing, Self-Publishing, E-books, Oyster, Scribd,

Big data's next frontier: Crowd-testing fiction (Credit: hocus-focus via iStock/Salon)

“Creative writing is dying, if not dead,” moaned one commenter, and he wasn’t alone. The story that provoked this particular episode of hand-wringing appeared in the New York Times’ Bits blog, and had to do with the ability of new e-book subscription services like Oyster and Scribd to track their members’ reading habits. If authors are armed with data about, for example, whether readers finish a book — and the exact point at which they abandon it if they don’t — then, wrote David Streitfeld, they “can shape their books to please their readers more.”

E-book retailers like Amazon and Barnes & Noble have been collecting information like this for a while now, but they keep it to themselves. Oyster and Scribd — no doubt aware that these days there’s much more money to be made from people’s fantasies of becoming successful authors than there is from actually selling books to readers — have said they are willing to share the data they gather, presumably for a fee. And authors, especially self-published authors, are very interested. Quinn Loftis, a self-published author of YA fiction who reportedly makes a “six-figure annual income” from her books, told Streitfeld, “If you write as a business, you have to sell books. To do that, you have to cater to the market. I don’t want to write a novel because I want to write it. I want to write it because people will enjoy it.”

That’s the sort of statement that inevitably cues a chorus of laments about the decline of Western Civilization in general and literature in particular. Not to mention the predictable dumb jokes about how it’s a good thing that some past great (Shakespeare or Joyce usually) isn’t around to see their work degraded by such fads.

The truth is that tailoring books to reader preferences has been going on for decades, and the Internet is only making this process more efficient. That doesn’t mean that, in the future, literary novelists like Donna Tartt or James McBride are going to be expected to market-research their books like a Hollywood filmmaker forced to submit his would-be blockbuster to the scrutiny of a test audience. While the editors who work with such writers are not above suggesting a sunnier ending or more intelligible plot points every now and then, for the most part, authors like these are signed on because of the originality of the work they produce. Only Donna Tartt can write a Donna Tartt novel, and it would take a very foolish editor to interfere overmuch with her process.



But a sizable sector of publishing doesn’t put a premium on originality, and not coincidentally, this is the same part of the book market that has been most transformed by the e-book self-publishing explosion. In genres like young adult fiction, romance, urban fantasy, thrillers, mysteries, spy novels and various fusions thereof, many readers prefer the familiar and the formulaic.

Romance publishing in particular has long made a science of creating pleasant, disposable fiction for readers seeking an escape from everyday life. Categories and lines of books were designed to look alike, despite having myriad authors, and aspiring romance writers were given outlines and lists of requirements native to particular subgenres (the heroine must be a virgin, the hero must be a duke, etc.). But romance isn’t the only form of popular fiction in which authors crank out made-to-order work; a friend of mine who wrote an interesting mystery set in a nursing home was recently sent a list of desired cozy-mystery genres by a publisher who thought she might be able to supply them with whodunits set among communities of scrapbook makers, pastry chefs and quilters.

In the past, publishers of mass-market paperback fiction had to deduce what their repeat customers expected from, say, a series about a kick-ass exorcist in leather pants and her adventures in the greater St. Louis area or a former Navy SEAL turned private eye solving crimes in San Diego. They and their authors got feedback from readers, librarians and booksellers about what went over well and what didn’t, but there was no systematic way to assemble and evaluate that feedback. There is an art (well, maybe a craft) to successful genre publishing, and much of it has been rooted in the intuition of editors and publishers about what readers want.

New e-book subscription services like Oyster and Scribd, with their limited inventory of top-shelf titles, are well-suited to such readers. You can’t get “The Goldfinch” or Eleanor Catton’s “The Luminaries” from Oyster, but you can gorge on a seemingly infinite array of erotic romances patterned after “50 Shades of Grey,” many of them self-published by writers who are eager both to know what readers want and to give it to them. But whether learning that most readers bail out halfway through their third chapter can help them much is uncertain. As Streitfeld pointed out, “Pride and Prejudice,” an incontestably popular novel, is finished by less than 1 percent of the Oyster subscribers who open it, but why? Possibly many of them simply chose it as a sample text when experimenting with the app’s features, while others, having already read the novel multiple times, really just popped in to revisit a favorite passage.

Writers who want constructive feedback from readers have better options. Many self-published authors use “beta readers” while revising their early drafts, a practice also common in fan fiction communities. Others post portions of their works-in-progress to blogs and gauge the response from their fans. With finished books, there are reader reviews on Amazon and social networking sites like Goodreads and Booklikes that can provide far more detailed reactions, even if the opinions aren’t necessarily representative or especially articulate.

Not coincidentally, the most collaborative (and contentious) reader-author interactions happen in the online communities that have grown up around the genres where formula flourishes. The world of young adult fantasy and romance fiction is one where the line between readers and authors can get pretty blurry. Readers, who often take credit for a favorite author’s success, feel entitled to complain loudly if they don’t like where a story is going, or where it ends. Fans of Charlaine Harris’ bestselling Sookie Stackhouse series vented their outrage all over the Internet when Harris gave her heroine a fate that was not to their liking. Threats were made, both personal and economic. “I won’t be investing anymore time or money in works by this author,” one reader wrote on Amazon. “We read fantasy for an escape, not disappointment.” Of course, she had to finish the book to arrive at that opinion, so within Oyster’s limited metrics, she might even register as a happy customer.

But no matter, because she wouldn’t have changed Harris’ mind in either case. As Harris’ agent told the Guardian, the author expected a certain faction of her fans to be deeply displeased with the conclusion of Sookie’s story and decided not to go on book tour to avoid being confronted by them. Still, she stuck to her guns. “I’m confident I stayed true to the characters I’d been writing for so many years,” the beleaguered author wrote on her Facebook page. If she can hold out, I’m pretty sure Shakespeare and James Joyce would have managed it, too.

Further reading

David Streitfeld for the New York Times on e-book subscriptions services tracking their customers’ reading

David Streitfeld for the New York Times on technologies that allow authors to tailor their books to meet readers’ tastes

Disgruntled fans threaten Charlaine Harris over conclusion to Sookie Stackhouse series

Laura Miller

Laura Miller is a senior writer for Salon. She is the author of "The Magician's Book: A Skeptic's Adventures in Narnia" and has a Web site, magiciansbook.com.

More Related Stories

Featured Slide Shows

  • Share on Twitter
  • Share on Facebook
  • 1 of 22
  • Close
  • Fullscreen
  • Thumbnails

    Once upon a time on the Bowery

    Talking Heads, 1977
    This was their first weekend as a foursome at CBGB’s, after adding Jerry Harrison, before they started recording the LP “Talking Heads: 77.”

    Once upon a time on the Bowery

    Patti Smith, Bowery 1976
    Patti lit up by the Bowery streetlights. I tapped her on the shoulder, asked if I could do a picture, took two shots and everyone went back to what they were doing. 1/4 second at f/5.6 no tripod.

    Once upon a time on the Bowery

    Blondie, 1977
    This was taken at the Punk Magazine Benefit show. According to Chris Stein (seated, on slide guitar), they were playing “Little Red Rooster.”

    Once upon a time on the Bowery

    No Wave Punks, Bowery Summer 1978
    They were sitting just like this when I walked out of CBGB's. Me: “Don’t move” They didn’t. L to R: Harold Paris, Kristian Hoffman, Diego Cortez, Anya Phillips, Lydia Lunch, James Chance, Jim Sclavunos, Bradley Field, Liz Seidman.

    Once upon a time on the Bowery

    Richard Hell + Bob Quine, 1978
    Richard Hell and the Voidoids, playing CBGB's in 1978, with Richard’s peerless guitar player Robert Quine. Sorely missed, Quine died in 2004.

    Once upon a time on the Bowery

    Bathroom, 1977
    This photograph of mine was used to create the “replica” CBGB's bathroom in the Punk Couture show last summer at the Metropolitan Museum of Art. So I got into the Met with a bathroom photo.

    Once upon a time on the Bowery

    Stiv Bators + Divine, 1978
    Stiv Bators, Divine and the Dead Boys at the Blitz Benefit show for injured Dead Boys drummer Johnny Blitz.

    Once upon a time on the Bowery

    Ramones, 1977
    “The kids are all hopped up and ready to go…” View from the unique "side stage" at CBGB's that you had to walk past to get to the basement bathrooms.

    Once upon a time on the Bowery

    Klaus Nomi, Christopher Parker, Jim Jarmusch – Bowery 1978
    Jarmusch was still in film school, Parker was starring in Jim’s first film "Permanent Vacation" and Klaus just appeared out of nowhere.

    Once upon a time on the Bowery

    Hilly Kristal, Bowery 1977
    When I used to show people this picture of owner Hilly Kristal, they would ask me “Why did you photograph that guy? He’s not a punk!” Now they know why. None of these pictures would have existed without Hilly Kristal.

    Once upon a time on the Bowery

    Dictators, Bowery 1976
    Handsome Dick Manitoba of the Dictators with his girlfriend Jody. I took this shot as a thank you for him returning the wallet I’d lost the night before at CBGB's. He doesn’t like that I tell people he returned it with everything in it.

    Once upon a time on the Bowery

    Alex Chilton, Bowery 1977
    We were on the median strip on the Bowery shooting what became a 45 single sleeve for Alex’s “Bangkok.” A drop of rain landed on the camera lens by accident. Definitely a lucky night!

    Once upon a time on the Bowery

    Bowery view, 1977
    The view from across the Bowery in the summer of 1977.

    Once upon a time on the Bowery

    Ramones, 1977 – never before printed
    I loved shooting The Ramones. They would play two sets a night, four nights a week at CBGB's, and I’d be there for all of them. This shot is notable for Johnny playing a Strat, rather than his usual Mosrite. Maybe he’d just broken a string. Love that hair.

    Once upon a time on the Bowery

    Richard Hell, Bowery 1977 – never before printed
    Richard exiting CBGB's with his guitar at 4am, about to step into a Bowery rainstorm. I’ve always printed the shots of him in the rain, but this one is a real standout to me now.

    Once upon a time on the Bowery

    Patti Smith + Ronnie Spector, 1979
    May 24th – Bob Dylan Birthday show – Patti “invited” everyone at that night’s Palladium show on 14th Street down to CBGB's to celebrate Bob Dylan’s birthday. Here, Patti and Ronnie are doing “Be My Baby.”

    Once upon a time on the Bowery

    Legs McNeil, 1977
    Legs, ready for his close-up, near the front door of CBGB's.

    Once upon a time on the Bowery

    Suicide, 1977
    Rev and Alan Vega – I thought Alan was going to hit me with that chain. This was the Punk Magazine Benefit show.

    Once upon a time on the Bowery

    Ian Hunter and Fans, outside bathroom
    I always think of “All the Young Dudes” when I look at this shot. These fans had caught Ian Hunter in the CBGB's basement outside the bathrooms, and I just stepped in to record the moment.

    Once upon a time on the Bowery

    Tommy Ramone, 1977
    Only at CBGB's could I have gotten this shot of Tommy Ramone seen through Johnny Ramones legs.

    Once upon a time on the Bowery

    Bowery 4am, 1977
    End of the night garbage run. Time to go home.

  • Recent Slide Shows

Comments

Loading Comments...