The geeks who saved Usenet

Google's restoration of digital history relied on a few heroes' packrat mentality and a mountain of decaying mag tapes.


The geeks who saved Usenet

On May 11, 1981, one Mark Horton, then a graduate student at the University of California at Berkeley, using the e-mail address “ucbvax^mark,” posted this message to the Usenet newsgroup Net.general:

Rusty is right (or is that “Rusty is Wright”?)
- we have ALL in our .ngfile so I tend to forget
this. ALL.ALL may or may not work, but
ALL certainly does. Mark

Then, the ancient Internet scribe added this ominous postscript:

I plan to make the change on Tuesday
unless something horrible happens.

Horton’s message was a response to a previous post, the intact original of which is now lost to history, from one “sdcarl!rusty,” aka Rusty Wright. With this incomplete fragment of a cryptic exchange, the history of Usenet, as we have it today, begins.

The message is the oldest Usenet posting in the 20-year archive, now searchable on Google. It’s the first of some 700 million posts that provide a record spanning the early history to the present of Usenet — the sprawling public bulletin board, composed of a vast hierarchy of newsgroups, that grew up alongside the Internet itself.

Granted, this message doesn’t exactly have the ever-quotable and historic ring of Alexander Graham Bell braying on the first telephone call, “Mr. Watson. Come here. I need you.” But it’s not the first Usenet message ever — it’s just the first one captured in this vast, yet still incomplete, archive of Usenet’s 35,000 topic categories. It’s an ordinary exchange between two of the first few hundred denizens of Usenet posting back in 1981.

Still, if you squint, you can see glimmers of what’s to follow in this poignant gem of a fragment. What are these geeks talking about, anyway? It’s a meta-post about the system itself, of course! It’s part of a technical discussion of how Usenet should be administered. And catch that corny play on words, goofing off Rusty’s last name: “or is that ‘Rusty is Wright’?”

Geeks talking amongst themselves on Usenet about how Usenet should best be run, while having fun with homonyms: Almost 20 years later, has anything really changed?

In mid-December 2001, Google unveiled its improved Usenet archives, which now go more than a decade deeper into the Net’s past than did the millions of posts that the company salvaged from DejaNews. Now on a browser near you: a glimpse of the prehistory of the Net culture we all take for granted today. The first “me too” post! The first “Make-Money-Fast” post! It’s enough to make even a relative newbie nostalgic for a past she never experienced firsthand.

The debut of the archive touched off a flurry of chatter among the geeks on Slashdot, some of whom had been there back in the day. There were some grumbles. Imagine what it’s like to see your flames from 15 years ago, when Usenet still had the population of a small town, now searchable by anyone on the Web.

“Glad I’ve changed my e-mail address since those long, (best) forgotten days. It wasn’t me, I swear,” joked one poster to Slashdot. Another one griped: “It’s like having naked baby pictures of yourself stapled to your forehead when you walk around.” (Google vows that at the author’s request, they’ll delete old posts; so if you want to be the Internet equivalent of a rare-book burner, go right ahead.)

Google gets the credit for making these relics of the early Net accessible to anyone on the Web, bringing the early history of Usenet to all. Michael Schmidt, 29, a Google software engineer, spent the last year and a half playing detective, trying to track down the Internet’s lost history: “It was a long and painful investigative process. I was searching on the Web, calling people. There were a lot of dead ends.”

But it was the geeky pragmatism and historical foresight of Usenet old-timers themselves that actually saved the early history of the newsgroups so that we can all poke around in it today. These “archive donors,” whom Google thanks here, gave their copies of the millions of messages they’d saved back to the Net.

The tale of how early Usenet was saved begins with one of the Net’s great old-timers: Henry Spencer. “Henry Spencer is the real hero, because his contributions are what makes this historic,” says Schmidt. “Back in the Stone Age of the Internet, he was already archiving this stuff, and he was the only one doing it.”

Spencer, a legendary Unix hacker — a species not exactly known for humility — is pleasantly understated about his role as Usenet’s great early archivist. He’s the first to point out that he wasn’t really the only one saving those early messages. But the copies he kept of Usenet postings from 1981 to 1991 appear to be the only ones that still exist. “There were several other people who were archiving stuff, but all of them gave up before we did, and as far as I know none of their archiving survived,” he says. For instance, legend has it that two guys at Bell Labs kept back-ups as well, but their stores of these ultra-rare posts are nowhere to be found.

“I’m very glad the stuff is finally out there, and I can stop worrying about how the only copy might get lost,” Spencer says, now that Google has assured the preservation of the more than 2 million old messages he saved. “I’m just glad that this particular great mass of data is no longer my worry.”

One of the early adopters of the computer language C, Spencer is known for his Ten Commandments for C Programmers, as well as for being the coauthor of C News, one of the early programs for transferring and reading Usenet messages.

Now 46 years old, he works as an independent consultant, but back in 1981 he ran the computer facility at the University of Toronto’s zoology department. While the geeks over in the university’s computer science department were busy with the Arpanet, the Department of Defense’s system was too expensive for the zoologists.

“The zoology department may sound like a funny place for pioneering networking work,” says Spencer. “But the computer science department wasn’t very interested in this inferior networking. It was very low-tech by their standards. But it worked and theirs didn’t. Their opinion changed fast when we started providing e-mail.”

That’s how, in the spring of 1981, with a 300 baud modem, the zoology department at the University of Toronto became a central distribution point for Usenet, when the network was just 2 years old.

Traffic was almost unimaginably lighter in those days. Only about 200 people had access to Usenet: “In the first few years, it was at least plausible to come in in the morning and read all the Usenet traffic that had come in, and 15 minutes later be off doing something useful,” remembers Spencer. But even that low level of traffic was too much for the storage requirements of the day. “Pretty soon, it was necessary to think about expiring old stuff,” he says.

It wasn’t a sense of historical importance that initially led Spencer to think about creating an archive. His motivation was much more pragmatic than that: Most of the conversations on Usenet at the time were very technical, and he was reluctant to see the information in them disappear, because it might be useful to the university’s geeks: “A lot of the early traffic was about things like Unix systems bugs, and it seemed unwise to just throw it out.”

So the archiving began with 40 megabytes filling up a new mag tape — each reel one-half inch thick and 10 inches in diameter — every few months. In this era, messages from the outside world came in at the tortoise rate of 300 baud. (“When we got a 1,200 baud auto-dialing modem, that was just wonderful. Twelve-hundred baud was just total luxury,” Spencer recalls.) As Usenet grew, this meant that Spencer and his system administrators had to be selective about which newsgroups they received and archived, keeping technical conversations but throwing away some of the more general discussions that generated a lot of traffic.

“We started dumping stuff that we thought was obviously of no future use, groups that specialized in a lot of talk and no substance, so to speak. For example, fairly early on there was a newsgroup about abortion which specialized in violent arguments.”

That’s why not only the very earliest Usenet posts, before Spencer started archiving in 1981 (Usenet began in 1979) but even some of the posts in the 1980s are still lost. It’s too bad; today, wouldn’t more of us rather see what was being said about abortion in 1984 than sift through the arcana of bug fixes in systems that have probably been long since retired? “It was perfectly reasonable from the viewpoint of stuff that we might want to use again, but a little sad from today’s viewpoint,” Spencer admits.

For 10 years, the nine-track mag tapes piled up, hanging in a huge rack at the zoology department’s computer facility. Finally, in the early ’90s, with the growth of Usenet outpacing the zoology department’s budget for $15-a-pop tapes, the general archiving project ended.

In the spring of 1991, Bruce Jones, then a grad student in the communications department at the University of California at San Diego, flew to Ontario at his own expense. He was writing his Ph.D. dissertation on the history of Usenet and was eager to get his hands on Spencer’s tapes.

The 141 tapes, most of which held 120 megabytes of posts, now lived at the University of Western Ontario, thanks to a road trip in the middle of the Canadian winter that David Wiseman, the university’s network administrator, had taken earlier that year to unburden the University of Toronto’s zoology department of them.

Jones would spend the next two weeks rescuing the data off them. Not only was the tape technology rapidly becoming obsolete — just try to find a working tape-reader today — but the tapes themselves do not have anything like a 10-year shelf life.

By now the historical import of the tapes was already apparent. But spending two weeks running tapes through a tape-cleaning machine and dumping them on disks was the prerequisite to even looking at them. “Spencer had written a program for removing data from tapes when the tapes went bad,” Jones explains. “I was just the first person who was willing to invest my time and money — a lot of people wanted to see what was on them.” In two weeks, Jones got through the first 105 tapes.

“Usenet has always been about arguing about itself,” Jones says of the posts that were unearthed. “And the arguments that you see today are the same arguments that go way back into the early ’80s, and I’m sure that those arguments will continue well into the future.”

Case in point: the fact that the older parts of the archive are now available on Google has given Usenet denizens something new to argue about. “I’ve already gotten three letters from people accusing me of trying to make money off these archives,” Jones observes wryly. All the “archive donors” gave the posts to Google for posterity.

Over the next 10 years, Wiseman got through the remaining three dozen or so tapes by wangling the time and energies of “bored graduate students.” But by 1995, constrained by university budgets, the archiving project was running out of disk space.

So, Brewster Kahle, the creator of the Web’s other major archiving project, the Internet Archive Wayback Machine, chipped in, donating a then-humongous nine-gigabyte hard drive to the cause.

In the end, they pulled more than 2,056,000 posts off the 141 tapes. “It took us 10 years. I got so busy and everybody else got less interested,” says Wiseman, almost sheepishly. More than 2 million posts: It doesn’t sound like a lot compared to the 700 million total in Google’s archive, but they’re the oldest remnants.

Apparently someone is still interested. Wiseman used FTP to hand off the files to Google. And just after Google announced the availability of the archive, some rogue used FTP to grab the whole archive off the University of Western Ontario’s FTP server — all three gigs of it transferring in one night. “I have no idea what they plan on using it for, since if it’s spam e-mail the addresses are all wrong,” says Wiseman. Now, anyone who wants a full copy will have to ask politely first — it’s no longer on the server.

Google filled in the more recent posts not covered by the old DejaNews archive thanks to J|rgen Christoffel of the German National Research Center for Information Technology, who’d kept his own archives in the ’90s, and Kent Landfield, a network security developer and the maintainer of

Landfield started archiving with entrepreneurial motives. In 1992 and 1993, while at Sterling Software in Omaha, Neb., Landfield had a side project that sold CDs of the Usenet archive. For $349.95 a year, every month you could get a CD burned with the content of Usenet. It was an attempt to cater to the user with a slower modem who still wanted access to every newsgroup.

“I realized that there was definitely a valuable historical aspect to the CDs themselves,” says Landfield. “The reality is, everybody thought that. We’re all just a bunch of packrats. We all knew there was a value to it, and it was a matter of how and when it would be used.”

Thanks to these packrats, Google now estimates that 95 percent of the posts ever made to Usenet are now searchable from the site. But Spencer, for one, can’t help thinking of all that’s still been lost — not just of the other 5 percent of Usenet, but also of the other early history of online communication.

Think of the Arpanet mailing lists that were the precursors to Usenet. Spencer points out that while most of the mailing lists kept archives, a significant number of them have been lost over time. “The first flame war, things like that, most certainly dates before Usenet,” he says. “And I would bet that a lot of that material is gone, because at some point, nobody thought it was worth saving.”

More Related Stories

Featured Slide Shows

  • Share on Twitter
  • Share on Facebook
  • 1 of 11
  • Close
  • Fullscreen
  • Thumbnails

    Ten spectacular graphic novels from 2014

    Beautiful Darkness by Fabien Vehlmann & Kerascoët
    Kerascoët's lovely, delicate pen-and-watercolor art -- all intricate botanicals, big eyes and flowing hair -- gives this fairy story a deceptively pretty finish. You find out quickly, however, that these are the heartless and heedless fairies of folk legend, not the sentimental sprites beloved by the Victorians and Disney fans. A host of tiny hominid creatures must learn to survive in the forest after fleeing their former home -- a little girl who lies dead in the woods. The main character, Aurora, tries to organize the group into a community, but most of her cohort is too capricious, lazy and selfish to participate for long. There's no real moral to this story, which is refreshing in itself, beyond the perpetual lessons that life is hard and you have to be careful whom you trust. Never has ugly truth been given a prettier face.

    Ten spectacular graphic novels from 2014

    Climate Changed: A Personal Journey Through the Science by Philippe Squarzoni
    Squarzoni is a French cartoonist who makes nonfiction graphic novels about contemporary issues and politics. While finishing up a book about France under Jacques Chirac, he realized that when it came to environmental policy, he didn't know what he was talking about. "Climate Changed" is the result of his efforts to understand what has been happening to the planet, a striking combination of memoir and data that ruminates on a notoriously elusive, difficult and even imponderable subject. Panels of talking heads dispensing information (or Squarzoni discussing the issues with his partner) are juxtaposed with detailed and meticulous yet lyrical scenes from the author's childhood, the countryside where he takes a holiday and a visit to New York. He uses his own unreachable past as a way to grasp the imminent transformation of the Earth. The result is both enlightening and unexpectedly moving.

    Ten spectacular graphic novels from 2014

    Here by Richard McGuire
    A six-page version of this innovative work by a regular contributor to the New Yorker first appeared in RAW magazine 25 years ago. Each two-page spread depicts a single place, sometimes occupied by a corner of a room, over the course of 4 billion years. The oldest image is a blur of pink and purple gases; others depict hazmat-suited explorers from 300 years in the future. Inset images show the changing decor and inhabitants of the house throughout its existence: family photos, quarrels, kids in Halloween costumes, a woman reading a book, a cat walking across the floor. The cumulative effect is serene and ravishing, an intimation of the immensity of time and the wonder embodied in the humblest things.

    Ten spectacular graphic novels from 2014

    Kill My Mother by Jules Feiffer
    The legendary Pulitzer Prize-winning cartoonist delivers his debut graphic novel at 85, a deliriously over-the-top blend of classic movie noir and melodrama that roams from chiaroscuro Bay City to Hollywood to a USO gig in the Pacific theater of World War II. There's a burnt-out drunk of a private eye, but the story is soon commandeered by a multigenerational collection of ferocious women, including a mysterious chanteuse who never speaks, a radio comedy writer who makes a childhood friend the butt of a hit series and a ruthless dame intent on making her whiny coward of a husband into a star. There are disguises, musical numbers and plenty of gunfights, but the drawing is the main attraction. Nobody convey's bodies in motion more thrillingly than Feiffer, whether they're dancing, running or duking it out. The kid has promise.

    Ten spectacular graphic novels from 2014

    The Motherless Oven by Rob Davis
    This is a weird one, but in the nervy surreal way that word-playful novels like "A Clockwork Orange" or "Ulysses" are weird. The main character, a teenage schoolboy named Scarper Lee, lives in a world where it rains knives and people make their own parents, contraptions that can be anything from a tiny figurine stashable in a pocket to biomorphic boiler-like entities that seem to have escaped from Dr. Seuss' nightmares. Their homes are crammed with gadgets they call gods and instead of TV they watch a hulu-hoop-size wheel of repeating images that changes with the day of the week. They also know their own "death day," and Scarper's is coming up fast. Maybe that's why he runs off with the new girl at school, a real troublemaker, and the obscurely dysfunctional Castro, whose mother is a cageful of talking parakeets. A solid towline of teenage angst holds this manically inventive vision together, and proves that some graphic novels can rival the text-only kind at their own game.

    Ten spectacular graphic novels from 2014

    NOBROW 9: It's Oh So Quiet
    For each issue, the anthology magazine put out by this adventurous U.K.-based publisher of independent graphic design, illustration and comics gives 45 artists a four-color palette and a theme. In the ninth issue, the theme is silence, and the results are magnificent and full of surprises. The comics, each told in images only, range from atmospheric to trippy to jokey to melancholy to epic to creepy. But the two-page illustrations are even more powerful, even if it's not always easy to see how they pertain to the overall concept of silence. Well, except perhaps for the fact that so many of them left me utterly dumbstruck with visual delight.

    Ten spectacular graphic novels from 2014

    Over Easy by Mimi Pond
    When Pond was a broke art student in the 1970s, she took a job at a neighborhood breakfast spot in Oakland, a place with good food, splendid coffee and an endlessly entertaining crew of short-order cooks, waitresses, dishwashers and regular customers. This graphic memoir, influenced by the work of Pond's friend, Alison Bechdel, captures the funky ethos of the time, when hippies, punks and disco aficionados mingled in a Bay Area at the height of its eccentricity. The staff of the Imperial Cafe were forever swapping wisecracks and hopping in and out of each other's beds, which makes them more or less like every restaurant team in history. There's an intoxicating esprit de corps to a well-run everyday joint like the Imperial Cafe, and never has the delight in being part of it been more winningly portrayed.

    Ten spectacular graphic novels from 2014

    The Shadow Hero by Gene Luen Yang and Sonny Liew
    You don't have to be a superhero fan to be utterly charmed by Yang and Liew's revival of a little-known character created in the 1940s by the cartoonist Chu Hing. This version of the Green Turtle, however, is rich in characterization, comedy and luscious period detail from the Chinatown of "San Incendio" (a ringer for San Francisco). Hank, son of a mild-mannered grocer, would like to follow in his father's footsteps, but his restless mother (the book's best character and drawn with masterful nuance by Liew) has other ideas after her thrilling encounter with a superhero. Yang's story effortlessly folds pathos into humor without stooping to either slapstick or cheap "darkness." This is that rare tribute that far surpasses the thing it celebrates.

    Ten spectacular graphic novels from 2014

    Shoplifter by Michael Cho
    Corinna Park, former English major, works, unhappily, in a Toronto advertising agency. When the dissatisfaction of the past five years begins to oppress her, she lets off steam by pilfering magazines from a local convenience store. Cho's moody character study is as much about city life as it is about Corinna. He depicts her falling asleep in front of the TV in her condo, brooding on the subway, roaming the crowded streets after a budding romance goes awry. Like a great short story, this is a simple tale of a young woman figuring out how to get her life back, but if feels as if it contains so much of contemporary existence -- its comforts, its loneliness, its self-deceptions -- suspended in wintery amber.

    Ten spectacular graphic novels from 2014

    Through the Woods by Emily Carroll
    This collection of archetypal horror, fairy and ghost stories, all about young girls, comes lushly decked in Carroll's inky black, snowy white and blood-scarlet art. A young bride hears her predecessor's bones singing from under the floorboards, two friends make the mistake of pretending to summon the spirits of the dead, a family of orphaned siblings disappears one by one into the winter nights. Carroll's color-saturated images can be jagged, ornate and gruesome, but she also knows how to chill with absence, shadows and a single staring eye. Literary readers who cherish the work of Kelly Link or the late Angela Carter's collection, "The Bloody Chamber," will adore the violent beauty on these pages.

  • Recent Slide Shows



Comment Preview

Your name will appear as username ( settings | log out )

You may use these HTML tags and attributes: <a href=""> <b> <em> <strong> <i> <blockquote>