Imagine the world's largest community -- a place where millions of people converse daily in civilized tones about everything from cereal to Camus. A place free of spam, porn, come-ons and flame wars. A place, in fact, where your thoughtful ruminations might actually be rewarded with prizes.
Is this some new, gated online space requiring an annual membership fee? Nope -- it's the new vision some entrepreneurs have for Usenet, the Net's oldest and most anarchic interactive zone.
The last year has seen a burst of products designed to tame Usenet into a consumer-friendly product: from the explosive growth of pioneer Deja News to more recent products like Talkway and Realize. These Web-based commercial services are implementing everything from personalization to collaborative filtering to make Usenet a warm, friendly place where novices will want to participate.
As Talkway founder Richard Simoni puts it: "We want to make Usenet really accessible and really approachable: We don't want it to be a techie haven where nerds like me can talk about their little nerd interests. We want it to be an actual consumer product."
Usenet was founded in 1979 as a bulletin board system for Unix programmers; today, by one count, there are at least 65,000 newsgroups, last month alone hosting nearly 10 million posts by 1.2 million people. Though technically Usenet exists as a sort of parallel network, socially it is the backbone of the Internet -- the most rudimentary yet inherently interactive community.
Yet for years people have been talking about its demise. It may be huge, but it's also a garbage-filled and abstruse system that hasn't changed much since its inception, and it certainly hasn't grown as fast as its baby sibling, the Web.
Consider the hurdles that newsgroup neophytes have to overcome: Until recently, reading Usenet meant downloading a client program, configuring it to communicate with one of the "news servers" that handles Usenet traffic and deciphering a mire of directories before eventually stumbling across a newsgroup that was both relevant to your interests and trafficked by humans. Then, of course, you would have to wade through the spam, flame wars and pornographic solicitations. Compare that to the relevance and ease of joining, say, a moderated forum on CNN.com, and it's no surprise that people have been predicting Usenet's death.
"My overall view on this is that Usenet, the newsgroups, never really evolved out of the old Internet interfaces," says Eric Horvitz, who is researching Usenet at Microsoft. "It's this arcane world still that's not really friendly to a majority of consumers. It's like the ham radio users, the CB users of the Internet."
But the new Usenet companies are convinced that Usenet can be saved from ham radio's fate. Since Usenet is a system without owners or real authorities, it's not likely that the infrastructure is going to change any time soon; but they believe that a commercial interface might be the key to morphing Usenet from niche market to mass market.
The cleanliness does comes at a cost. After all, these are all commercial services, invented by companies that firmly plan on making a profit from advertisements and e-commerce and premium subscription services. There are certainly people who like Usenet's wild and woolly ways, and wouldn't like to see the chaos tamed by profit-oriented companies.
But the advantage of their approach, the companies say, is that they are working with the interface, not within Usenet itself -- you'll always have the option of dredging up an old-fashioned newsreader and facing Usenet without filters and ad banners. As Simoni puts it, "Usenet has been seen as a tough nut to crack: You can't post ads in there without getting flamed, and the culture of the past was that commercial activity was discouraged. But we're getting around that by keeping the commercial activity within our service and not putting it in Usenet proper. That way everyone gets the best of both worlds."
Deja News has led the way in bringing Usenet to the masses. It revolutionized Usenet by archiving conversations back to two years (formerly, few servers archived more than two weeks) and making them all available via a zippy Web search engine. But Deja News' growth exploded at the end of 1997, when the company retooled itself as more of a personalized "discussion network," allowing people to post to Usenet via the Deja News Web site. Today, Deja News claims more than 5 million users.
Without a doubt, Deja News is attracting a whole new audience to Usenet. In fact, many of the people who now use Deja News aren't even aware that they are reading Usenet: Many just think they're reading the discussion forums of Deja News. "I would say that about half, maybe a little bit more, of our users have also accessed Usenet by other methods, using software of some kind," says David Wilson, vice president of marketing for Deja News. "But the other approximate half use Deja News only. And it's been moving more and more towards Deja News becoming the sole access point for our users."
Marc Smith, who tracks Usenet usage with his Netscan project, says that Deja News has had a tremendous impact on the sheer number of Usenet posts, but the ease-of-use in fact hasn't necessarily fomented more community. "The [time] cost used to keep certain people out and certain people in, because after you'd put time and effort into a group it was hard to transfer that elsewhere," he says. "These tools allow you to walk up to Deja News, type something in and messages pop out. You don't even know anything about the group; you find the message, get the value and leave. It's the difference between a small town and a city: I'm more anonymous and my connection and ties to the group are curtailed."
Deja News has provided some personalized tools to try to encourage community growth: offering a "My Deja News" service with spam-free e-mail accounts and personal hot lists, and allowing users to start their own forums. But perhaps the biggest hurdle to encouraging users to get involved in the Usenet community is the quality of discussion itself. Who, after all, wants to plunge into newsgroups that are peppered with flame wars, puerile arguments and pointless questions and dominated by a few people on soapboxes?
This is the issue that the new collaborative filtering tools, like Talkway and Realize, are trying to solve. Their solution is, simply, to raise the bar on conversation.
Like Deja News, 4-month-old Talkway is a Web-based entrance to Usenet, with personalized accounts and a search engine plus a Yahoo-like organizational system that converts the official names of newsgroups (say, "comp.sys.mac.general") into comprehensible ones ("General information on Macintosh"). Talkway has brought in commerce partners, so an array of buttons allows you to buy products that are related to certain newsgroups. But Talkway also allows users to self-police their newfound communities by voting on the quality of each post; idiotic posts can, quite simply, be filtered right out of the conversation.
Part of this system is straightforward: Your personalized account allows you to filter out posts by person or keyword, so that you'll never have to see the posts by that irritating bigot or read debates about a subject that doesn't interest you. Talkway has also, however, implemented a complex collaborative filtering system; members can rate the quality of posts as spam, adult, flame or great -- if enough people vote a message as, say, "flame," and you turn your filter on to eliminate all flames, then you'll never see that message.
"There are so many people and so much information in Usenet that it's hard to deal with the infoglut," explains Richard Simoni, Talkway co-founder. "There's different ways of dealing with that problem, and one way to do it is leverage the value of the community itself to help you decide what's valuable to you."
Why would someone want to take the time to rate messages, though? In comparison to other text-based Usenet newsreaders -- which despite their complexity are often quick to load -- most of the Web interfaces sacrifice speed to a friendly graphic design: Not only does it take time to load and scroll through posts, but you have to pause to vote on each message.
Realize, another Usenet product that launched in early September, has a collaborative filtering system similar to that of Talkway, but is trying to get people to contribute ratings by offering incentives. Realize -- which only deals with the 50 most popular newsgroups -- rewards posters with Qy Points, which can be redeemed for frequent-flier miles, product discounts or donations to charity if they demonstrate "valuable contributory behavior to online message boards." It's the virtual equivalent to a gold star: If you rate lots of messages or if your messages are rated positively by other members, you are rewarded with points.
But the primary flaw in collaborative filtering is that it requires a critical mass to work: Until many people are consistently participating in the system, filtering results will be spotty. Realize's system requires six to nine ratings before a "bad" message is filtered out; additionally, any unrated messages are automatically filtered out. So unless a good number of people are diligently rating all messages -- turning their filter off, in fact, so that they can see the new, unrated messages -- no one using the filter will be able to view anything at all.
Although Talkway and Realize claim to have 100,000 and 10,000 users respectively, both admit that their systems don't work right yet, and won't until they have more participants. After all, 100,000 users is a paltry number when compared to the vastness of the millions of Usenet messages that each need multiple ratings.
But if collaborative filtering does work, what kind of impact could that have on Usenet discussions? "Rating messages" is a certain kind of censorship -- albeit one that is less top-down than the "cancelbots" or automated programs that have long roamed Usenet with orders to kill spam and porn. Fringe or controversial posts might be whisked out of view by a system that relies on lowest-common-denominator voting. What, for example, constitutes porn? All it could take is 10 conservative voters to decide that a frank discussion about sex should be hidden from view.
Then there are going to be the mavericks who bend the rules. Explains Marc Smith of Netscan, "It's a problem if we have one huge pot of collaborative filtering, because we will have opinion without accountability. No one will know who voted what, and that means that people will spam the filter. If it becomes the case that filters get critical mass and change what is seen, those who want to be seen will go out and vote until they are seen." Or vice versa: Enemies could potentially "kill" each others' posts by voting multiple times. As Smith puts it, "If you want to live in a Disney-filtered universe, OK. Just don't make me live in it."
The collaborative filtering companies are aware of these pitfalls, and emphasize that you can turn the filters off, if you like, to check up on what you're missing. But will people take the extra time to double check that they aren't losing out on a mind-expanding discussion?
Eric Horvitz of Microsoft, who is also researching a collaborative filtering system for Usenet, is concerned about just this: "A bad collaborative filtering solution might hide interesting parts of the Internet because users will come to rely on the tool. At Microsoft we're interested in probing pitfalls of various kinds. We're looking at techniques of expanding what you might be looking at instead of shrinking and focusing it. There's always a danger [the filters] might not take you to a place you would want to go if you had time to think about what you wanted. There are tremendous trade-offs."
Despite these hurdles, the new Usenet companies believe that their system could eventually transform Usenet into a less chaotic and more relevant community. Michael Ginn, president of Realize, postulates: "Our hope is over the long term we can help catalyze a shift in Usenet from a completely anarchistic system that sometimes does or doesn't work to a system that naturally and organically tends to address the interests of people within the group."
And if the promise of a more approachable Usenet draws more people into the fray, Usenet could conceivably grow even vaster than it is today. More people might be willing to join in if they can contribute in small ways -- by voting and quietly helping shape a group -- instead of having to post copiously in order to participate.
As Smith puts it, "This could be the second coming of Usenet."
Shares