Too much of a good thing

Forget spam -- our real conundrum is the overload of legitimate e-mail. But help is coming.

Published July 16, 2004 7:29PM (EDT)

Early in June, Lawrence Lessig, a professor at Stanford Law School, found himself confronting an extreme version of a problem familiar to many of us -- an overflowing e-mail box. Lessig, a polymath whose sharp critiques of copyright law have made him famous online, receives a great deal of mail. On a typical day, between 100 and 200 messages (not counting spam) will crash into his in box, and, incredibly, he manages to keep up with most of it. But over the years, one of Lessig's mail folders -- a box called Reply To, stuffed with messages from strangers he felt deserved some kind of response -- had ballooned to intolerable proportions. By June, Reply To contained almost a thousand messages. That's when Lessig had an epiphany. "I realized I wasn't ever going to be able to reply to it all," he says. "I have a son who's 10 months old. I saw that I could spend the time answering e-mail, or I could spend the time with my son. So that's what motivated me to do it."

What Lessig did wasn't very novel -- he gave up. Answering the mail in his Reply To folder was going to take time, and he had better ways to spend it. But where you or I might simply have deleted the waiting messages, Lessig decided to appeal to the Karmic Gods of the Internet. "Dear person who sent me a yet-unanswered e-mail," Lessig wrote in a rueful form letter to each of his would-be correspondents. "I apologize, but I am declaring e-mail bankruptcy." Under the terms of this "bankruptcy," Lessig explained, he would ignore all the messages in his brimming folder, but he would allow the senders to write back to him if they really, truly wanted to get his attention. "It was an extraordinarily liberating act," Lessig says now. He mailed out hundreds of the bankruptcy notices, and only about 30 people sent back further missives. "I cleared those within a week."

Even though it worked pretty well, Lessig's declaration of "e-mail bankruptcy" as a response to e-mail overload is the kind of thing a psychotherapist might call a "coping mechanism" -- a desperate effort to gain a measure of control over a medium that increasingly feels beyond all control. Lessig is not alone in searching for some way to cope: These days we are all constantly bombarded with more messages than we can reasonably be expected to deal with, and in the absence of any perfect way to manage the mail, we've all kludged together ad-hoc methods of surviving.

The problems with e-mail go far beyond spam; indeed, for all the grousing over spam, getting rid of easily identifiable unsolicited commercial mail is probably one of the easier tricks engineers will pull off in the years ahead. The real crisis is with the legitimate mail -- we're drowning in it. Every day, dozens or hundreds of people -- people we want to keep in touch with -- use e-mail to make demands on our time. And it's not only the volume of mail that's killing us, it's also the variety. If you're a "knowledge worker" -- and who isn't a knowledge worker these days? -- just about every facet of your life is tracked by e-mail. You use mail not just to communicate, but to order and organize your existence: E-mail's where we keep our travel itineraries, our tax forms, our doctor's appointment reminders, our wedding pictures. If the cops were ever to ask you what you were doing on the night of so and so, where, besides e-mail, would you look for your alibi? (And just ask Henry Blodget or Martha Stewart where the cops would look for incriminating evidence.)

But if e-mail is so good, why does it feel so bad, especially for those of us who send and receive a lot of mail? Why can't today's dominant e-mail programs (such as Microsoft's Outlook or Qualcomm's Eudora) automatically prioritize your messages in your in box, or easily search for one old message hidden in a stash of hundreds of thousands? Why, instead, do we need to construct elaborate triage strategies -- sorting, filtering, filing, redirecting, etc. -- just to make sure we don't miss anything important? And, despite these, why do we still so often miss what is important, and why are we bombarded by the trivial? Why, most fundamentally, must we constantly work on our e-mail, vigilantly imposing our own schemes of order upon the incoming chaos, constantly guarding against getting behind, against the shame of e-mail bankruptcy?

The obvious weaknesses of e-mail have led many experts over the years to predict that e-mail's end is nigh, and today, tech leaders routinely pronounce e-mail dead. But the truth is not so dire. A host of companies, among them Google, have recently introduced some very novel e-mail programs, and are determined to make e-mail a little easier than it is today. They'll probably never make it perfect, but help is on the way.

In 1971, in a lab in Cambridge, Mass., the computer engineer Ray Tomlinson invented e-mail. By 1972, we can imagine, users of the new system might have been experiencing two novel, dissonant sensations: Endless joy at the ease and power of e-mail, and frustration at how badly it did everything they wanted it to do. We might like to think that our current problems with e-mail are new, caused by insidious "network effects" -- basically, by the fact that so many people are now online and e-mailing. But e-mail was ever thus: For almost two decades, researchers have been conducting "field studies" on e-mail users in office settings. In every study, even during the earliest days of e-mail, when mail volume was a fraction of what it is today, "active e-mail users" have reported feeling overwhelmed by mail, critically close to losing all control. The pain we feel in e-mail is nothing new.

What's curious, though, is that there is no set number of messages at which people begin to feel adrift with e-mail -- depending on personality and work habit, some people fall apart at 20 messages in a day, while others can deal with 200. This goes to a deeper fact about e-mail, one that explains many of the shortcomings of today's e-mail programs: Everyone uses e-mail differently. There is no accepted way to use e-mail; nobody ever tells you how to use it, or what to use it for. This is remarkable, since most of us use most products in similar ways -- each of us operates an automobile or a Web browser or a cellphone in just the same way as everyone else. E-mail is a fundamentally different kind of application. Once we start using it, we all end up working with e-mail in a highly personalized manner.

The wide variety of ways in which people use e-mail was first discovered in 1988 by Wendy Mackay, an MIT researcher who studied two dozen "extremely active" e-mail users at a large corporate research lab. By tracking how people worked with e-mail, Mackay found that there are a few classic e-mail user "types," the two most memorable of which were the folks she called "prioritizers" and those she labeled the "archivers." Prioritizers are people who manage their mail by simply not paying very much attention to it. For them, e-mail is just a tool at the workplace, like the phone or the fax machine -- it helps you do what you need to do in order to get your main job done. "My goal is to read as little mail as possible," a prioritizer named Mary told Mackay. "I try not to read mail more than once a day; I budget my time." Mackay wrote that Mary's colleagues were often frustrated because she did not always respond to her mail. Mary's defense was that if people really needed her urgently, they'd call her on the phone.

The more interesting e-mail personality belongs to the archivers, the people who save every piece of mail that comes to them, and who think of e-mail not just as a tool to help them with work and life but as a proxy to life itself. In Mackay's study, one archiver, a hapless computer scientist named Ralph, typified the breed. Unlike Mary, Ralph thought of mail as "an absolutely essential communication medium for both his job and his personal life." But Ralph's e-mail was a mess; his in box, Mackay wrote, was "always a jumbled mix of unseen messages, unclassified messages, and messages that remind him to do something." Ralph was unable to delete any messages out of a fear that he might get rid of something important: "What percent of the ocean don't you like?" he asked Mackay.

This wasn't an idle fear on his part -- Ralph actually did use the messages in his in box as a guide to what tasks he needed to perform at work, and deleting e-mail could have been dangerous. As messy as it was, his e-mail outlined some kind of rough order for his life. And although Ralph did at times try to organize his messages into folders, he could never come up with a good scheme to help him find the messages later. He recognized that he desperately needed help -- "he feels as if the situation is completely out of control," Mackay wrote.

It would probably be pleasant to be a prioritizer like Mary, unconcerned about missing important messages, happy to spend just a few minutes a day with e-mail, content to delete whatever you no longer find useful. In the digital age, though, Mary's attitude toward mail seems a tad quaint. The key problem with e-mail today is that, as the medium has become more and more central to our lives over the years, we've all been pushed -- some of us against our wishes -- toward Ralph's position, characterized by a need to save every message we get. You don't have to be a journalist to consider it verboten to delete a single piece of e-mail. There are a host of professionals for whom this is true today -- computer programmers, academics, attorneys, financial experts, Web designers, bloggers, eBay salespeople, and probably dozens of others. And in the future, as our jobs become more tightly centered on communication, the archivers' way of life will surely become even more dominant.

In 2001, after extensively studying a group of e-mail users in various high-tech settings, Nicolas Ducheneaut and Victoria Bellotti, two researchers at Xerox's Palo Alto Research Center, coined a name for this emerging world of e-mail archivers: E-mail as Habitat. In a report under that title, they wrote that e-mail was not just another computer program. For many people, e-mail functioned as a place to "live" -- a central spot from which to manage their affairs.

In many ways, the trouble with today's e-mail programs is that they make poor habitats. Outlook and Eudora and programs like them can be nice to visit, and prioritizers like Mary find them quite useful. But living inside these programs is tough. Essentially, they're fixer-uppers: You need to do a lot of tweaking to each program in order to convince it to accommodate an archiving life, one in which e-mail is not just for communication but also for storing data and, most important, for managing tasks. This is not to say that the programs are broken, or that many people don't find them incredibly useful. There are certainly many Eudora and Outlook partisans in the world who've set up their programs to perfectly manage their digital lives. But that's somewhat beside the point. Shouldn't your e-mail client accommodate your life right out of the box?

The most basic problem with Outlook and Eudora is that they impose on users the idea of organizing messages into many distinct folders. Folders have been around since the earliest days of e-mail (they are, of course, inspired by the paper-based world of physical filing cabinets), and users have grown particularly fond of them over the years. There is something intuitively comforting about putting a message into a folder -- you're giving it a place, and putting something in its place feels clean, organized. Perhaps for this reason, Eudora and Outlook aggressively push folders. When I asked her what Outlook users should do to keep from feeling frustrated with their e-mail systems, a Microsoft spokeswoman said it was simple -- they should file their messages into folders.

Bill Ganon, the vice president for Qualcomm's Eudora Products Group, was even more enthusiastic about folders. "We are believers in the folders and files, in taking a message and putting it where I know it needs to be," he said. Indeed, Ganon likes folders so much that he recommends that Eudora users set up filters that automatically sort incoming mail into separate folders, bypassing the main in box.

This advice, though, is exactly antithetical to the strategy that most active e-mail users employ to tackle the mail. In 1996, Steve Whittaker and Candace Sidner, scientists at the software company Lotus, studied how fellow employees at the firm used their e-mail programs. What they found surprised them -- many people kept a great deal of their messages in their main in box rather than filing them into separate folders. For the Lotus employees they observed, the average number of messages stored in the in box was over 2,000, whereas just under a thousand messages were filed into separate folders. Why were the e-mailers keeping so many messages in the in box? The answer will seem obvious to you -- people use their in box for all their current tasks, which they want to keep readily accessible in one place rather than scattered in a dozen areas. They use the in box as a to-do list, or as a place for all the unfinished conversations they're participating in, or for those all too frequent messages of "indeterminate status," which contain information one isn't sure what to do with, or desire some as-yet-unformed reply. Filing messages into folders contributes to an "out-of-sight, out-of-mind problem," says Nicolas Ducheneaut, the e-mail researcher at PARC. "They're not in your in box anymore so you don't see them anymore, so you're actually missing stuff. The filing paradigm is not well suited to keeping track of tasks."

There is another problem with filing messages into folders -- it's time-consuming. Coming up with the correct filing scheme (does it make more sense to file messages by sender, or by project, or by date?) requires a lot of thinking, and sticking to that scheme requires a lot of work. In Whittaker and Sidner's study, it was often the most junior Lotus employees, people who tended to receive the lowest volume of mail, who filed diligently. The managers, people who received the most mail, were the least likely to spend a lot of time filing their messages. The managers expressed no guilt about not filing, either, because they realized what many e-mail beginners do not -- that filing doesn't really make finding messages any easier in the future.

If filing is so useless, why do the main e-mail firms hold it in such high regard? Part of the reason is that, for the longest time, software companies could offer no real alternative to filing as a way to organize mail. In the absence of any kind of filing scheme, the only way to find a specific message in a huge stash of e-mail is to perform a keyword search for it. But searching is computationally demanding, and only recently have PCs begun to offer the power necessary to perform rigorous searches of e-mail.

But search wasn't hampered only by the inabilities of the PC, it was also set back by users themselves. This might be hard to believe, but not too long ago, few computer users felt very comfortable with the idea of "searching" for data. On a PC, searching for something -- an old document, say -- was a chancy affair. There was no guarantee that you'd find what you were looking for, and to have any expectation of success you would probably have to enter in some abstruse bits of Boolean arcana -- AND, OR, etc. Today, of course, people know how to use search -- "it's a new skill that they have," notes Gina Venolia, a user interface researcher at Microsoft Research, which is the company's long-term R&D unit. "People have gotten used to the string-a-bunch-of-keywords-together approach, now that Google is a household name."

On April 1, Google announced that it was creating a Web-based e-mail service that would put a search engine at the center of the users' e-mail experience. Perhaps the only person in Silicon Valley wishing that the announcement was a particularly unfunny April Fools' joke was Raymie Stata, the founder and CTO of Stata Labs, which first unveiled its search-focused e-mail system more than a year ago. Stata Labs makes a product called Bloomba, an e-mail program meant to be a faster, more useful alternative to Outlook or Eudora. Stata, a young computer scientist who once worked on the Alta Vista search engine, conceived of the system after noticing a couple of things: how well users took to the idea of searching on the Web, and how poorly the dominant e-mail programs searched through their mail. "Outlook," Stata says, "is a fundamentally broken product." Searching for a single message out of thousands "literally takes forever" -- which is to say, it can take minutes, which on a computer might as well be literally forever. In Bloomba, which indexes all of your e-mail in a database for fast searching, searching for a message takes seconds, as fast or faster than Google will return a Web search.

Bloomba is an excellent mail program. It is fast, pretty and intuitive, and when you use it it becomes clear that the program has been especially designed for people who get a lot of e-mail, and for whom setting up an elaborate set of folders and filters is not a fun task. This model can be a bit hard for some people to get used to at first, Stata says; when people start to use the system, their natural instinct is to build a set of folders in which to store their messages. Bloomba does allow you to create folders, which is probably a good thing -- e-mail users have been so steeped in the idea of putting stuff into folders that doing away with them might have freaked people out. But the software "subtly discourages people from building carefully managed hierarchies," Stata says, and when people reach the second or third day of working on Bloomba, they arrive at what Stata calls an "epiphany point." Users suddenly realize that they can trust the search engine, that they really can forget about filing. There is a strange sense of liberation associated with this moment; when you don't have to worry about filing, when the guilt of keeping all your messages in one big stash is wiped away because you now have access to a fantastic search engine that can find anything you want, e-mail can seem pretty pleasant. You might even call it fun.

One feels a similar thing with Gmail, Google's Web-based mail program. Conceptually, Gmail, which is still in an invitation-only, beta release, works a lot like Bloomba. It discourages folder hierarchies and encourages users to get at their messages through the search engine. What's amazing about Gmail is that even though it's a Web-based application, it is lightning fast -- as fast or faster than every desktop mail program, including Bloomba. And Gmail features one of the first truly novel innovations in an e-mail interface to come along in a long while, "conversation view." The system presents a conversation thread -- a group of back-and-forth e-mail messages between you and your boss, say, or a 65-message set from a particularly boisterous mailing list -- into a single visual pane, allowing you to read every e-mail in its proper context. Actions can be performed on entire conversations instead of on single messages -- so you can delete or archive those 65 mailing list messages with a single click.

Conversation view is not an entirely new concept; e-mail interface researchers have been calling for its introduction for years. Gina Venolia, at Microsoft Research, and Nicolas Ducheneaut, at PARC, have even built and tested working prototypes of e-mail systems that employ this concept. So why is Google the first company to release it? "Because it's fresh code," says Venolia. "Once you start thinking of e-mail from the ground up, you can do a lot of these kinds of things."

Because Google and Bloomba don't have to worry about offending an installed base of users, they can do radical things like suggest an end to mail folders. For Microsoft, which has to constantly guard against offending the sensibilities of the almost 400 million worldwide users of Outlook, change must come more slowly. Venolia -- who is personally a fan of the no-filing, always-searching model of e-mail organization used by systems like Bloomba and Gmail -- suggested that future versions of Microsoft's e-mail programs would likely offer some version of an improved search, and possibly some improvements in the user-interface akin to conversation view. She noted that Longhorn, the code name for Microsoft's upcoming version of Windows, would likely be the place for those improvements. (Current users of Outlook who can't wait so long for a better search engine for their e-mail ought to consider trying an indexing program such as Lookout or X1. These programs essentially transform Outlook into the same sort of program as Bloomba or Gmail -- an e-mail client with a very fast search engine.)

It would certainly be nice if, after remaining practically frozen for so long, e-mail became a hotbed of innovation in the near future. The pace of improvement in the Web browser was fantastic during the 1990s' fight between Netscape and Microsoft, notes Brad DeLong, the University of California at Berkeley economics professor and popular blogger -- and since he receives about 200 messages a day, he hopes we'll see a similar competition between firms over e-mail.

Besides the elimination of file folders and the increased reliance on search, what else can make e-mail easier to use? For several years, Kaitlin Duck Sherwood -- author of the "Overcome Email Overload" series of books and the resident e-mail expert at the Open Source Applications Foundation, a group working on a much anticipated "Personal Information Manager" program called Chandler -- has been calling for one big change in e-mail clients. She wishes that programs would display your mail in the in box according to each message's priority rather than chronologically. When people look at an in box full of new messages, they don't read the messages serially, from newest to oldest. Instead, most people make a rough calculation of each message's priority, and they pick and choose which messages to read first -- for instance, people will look at e-mail from folks they know before looking at something from a stranger. Wouldn't it be nice if your e-mail client listed the messages from people you've corresponded with above all the messages from people you haven't spoken to? Sherwood wonders. Although it is possible, after a lot of work, to make Eudora and Outlook do this, "no client that I know of does this out of the box," she says.

There's no question that such an improvement would make e-mail more manageable. Visionary research projects like IBM's ReMail also promise to make the in box easier to deal with. What's not clear, though, is how much these technical improvements will help. Because for all that engineers tinker with the software, it could be that they'll really fix nothing much if the people who use e-mail still use it cluelessly, both on the receiving side and on the sending. This goes to a mystery over our frustration with e-mail -- when we say that we are overloaded with e-mail, are we complaining about bad software, or about bad people? Take spam, for instance. Is spam fundamentally a social and economic problem, or is it a technological problem? Should we solve spam by tinkering with legal code, or with computer code, or with both?

Clearly, we haven't figured these things out yet -- but the smart money seems to be on the idea that e-mail will be hard to work with unless we develop some way of making the humans who use it more capable with the system. "If you start a new job, nobody ever sits you down and says, 'You're going to be receiving a boatload of e-mail, here are some effective strategies,'" notes Rael Dornfest, the CTO of O'Reilly Media, and a man who receives more than 300 e-mail messages each day.

In the same way, nobody ever tells people what is and what is not acceptable to send via e-mail. "There's not yet well-developed norms about how people should be using it," Lawrence Lessig says. "I get messages from students around the world who'll say, 'I'm doing a paper about copyright -- can you tell me what I should talk about?' That's on the end of totally inappropriate. Then there's the opposite end, when someone says, 'Look at this, it's cool.' Those things are greatest, especially because those people know enough to write two lines. Then there are these people who write these thousand-word-long e-mails, they always start off with, 'My name is ... ' -- as if it didn't say that right on the e-mail. And then they'll explain everything in their lives. You need a clear way to signal to them this is not appropriate."

"The problem I was complaining about was not tech-based," Lessig adds. "I've done a lot to try to improve my interaction with the technology, but my problem is not something that Gmail will solve. It's just substantive -- too many people making demands on my time."

For Lessig -- and for the rest of us, who will someday soon be in the same boat as Lessig -- the only real solution is to shut himself off, to ignore great chunks of e-mail from strangers simply because he has no time to deal with it. Clay Shirky, a professor at New York University who has thought a lot about the future of e-mail, says that we'll all develop approximate filtering mechanismsm in which we'll make determinations like, "The bulk of these thousand messages are so low-yield that there might be one message I might like to read here, but I'm willing to lose that one to make it easier."

"It makes me sad," Shirky says. "There are people like John Perry [Barlow] whose life revolves around the odd happenstance contact that e-mail can provide. It saddens me enormously to think that making e-mail unbroken is going to create a loss of that kind of value."

By Farhad Manjoo

Farhad Manjoo is a Salon staff writer and the author of True Enough: Learning to Live in a Post-Fact Society.

MORE FROM Farhad Manjoo

Related Topics ------------------------------------------