By the time Bill Joy arrived in Berkeley, Calif., in 1975 to attend graduate school, the fabled capital of leftist radicalism was a bit ragged around the edges. If the 21-year-old programming wunderkind had glanced at the headlines blasting out of the local alternative weeklies, he might have wondered just what kind of insane mess he had gotten himself into. In San Francisco, Patty Hearst was on trial for a bank robbery committed while the newspaper heiress was toting machine guns for the Symbionese Liberation Army. In Oakland, the Weather Underground botched a bombing of a Defense Department building. Even the reliable bugaboo of CIA recruitment on the University of California's Berkeley campus failed to generate more than a token protest.
Berkeley was burned out, its radical energy wasting away in infantile terrorism, conspiracy theorizing and drug overdoses. The Free Speech Movement that had galvanized the university in the '60s belonged to another geological age. Ken Thompson, co-creator of the Unix operating system, graduated from Berkeley in 1966 with a degree in electrical engineering. He returned to the university from Bell Labs for a sabbatical in 1975. But the campus on which he had once walked to class through clouds of tear gas had changed. That year, says Thompson, Berkeley "had turned into the most politically apathetic place I'd seen."
But it was the right place for Joy. "He never looked at those [alternative] papers," says John Gage, a close friend of Joy's during the Berkeley years and later at Sun Microsystems, a company co-founded by Joy. Today, Joy calls himself a "staunch Democrat" and has recently carved out a new niche as a techno-skeptical doomsayer, but in the '70s he was, by his own description, "not an activist." Joy chose to attend UC-Berkeley instead of Stanford or MIT not because he was attracted by its politics or countercultural reputation but because the computer science department's hardware was so obsolete that he figured he'd have no choice but to confine his research efforts to studying computing theory -- which was exactly what he wanted to do.
But theory turned out not to be Joy's forte. He started hacking code and never stopped. "His goal was to build something that worked," recalls Gage. And so he did. During his seven years at Berkeley, Joy and a few other graduate students and staff researchers spearheaded an intensive software development effort that culminated, most famously, in a radically improved version of AT&T's Unix, known simply as Berkeley Unix or, more commonly, as BSD,* for Berkeley Software Distribution.
Talk about your killer apps! Berkeley Unix worked so well that DARPA* chose it to be the preferred "universal computing environment" linking together Arpanet* research nodes, thus setting in place an essential piece of infrastructure for the later growth of the Internet. An entire generation of computer scientists cut their teeth on Berkeley Unix. Without it, the Net might well have evolved into a shape similar to what it is today, but with it, the Net exploded.
How did the small group of Berkeley programmers pull off such a feat? Well, for one thing, there was Joy, a programmer around whom legends accrue like so many iron filings stuck to a magnet. He could read at age 3, play chess at 4 and, during his oral exams, invented a "sorting algorithm"* on the fly that so stunned his examiners, one of them later compared the experience to "Jesus confounding his elders."
But too much focus on Joy, a favorite target for business magazine hagiography, obscures the larger picture. Berkeley's most important contribution was not software; it was the way Berkeley created software. At Berkeley, a small core group -- never more than four people at any one time -- coordinated the contributions of an ever-growing network of far-flung, mostly volunteer programmers into progressive releases of steadily improving software. In so doing, they codified a template for what is now referred to as the "open-source software development methodology." Put more simply, the Berkeley hackers set up a system for creating free software.
BSD itself wasn't originally free software. Joy sold it, with the University of California's blessing, at a nominal cost only to people or institutions that had already purchased licenses permitting them access to the source code of AT&T Unix (although, in practice, Joy's efforts to verify whether would-be buyers really did own licenses may not have been overly vigorous). But in spirit, Berkeley Unix was indeed free: As Dennis Ritchie, Thompson's collaborator in creating Unix, observes, anyone who wanted to hack on Unix usually had access to the source code, one way or another. And if those hackers sent their modifications to Berkeley, and they were deemed good enough, they became part of a code base maintained by programmers who wanted nothing more than for their software to be widely used, for as low a cost as possible.
Berkeley Unix has morphed through multiple phase shifts since its inception some 20 years ago, from the Joy-dominated era of the late '70s and early '80s to the more collaborative period that began after Joy's departure to Sun in 1982. But in the early '90s, after a bitter confrontation with AT&T, BSD finally did become "freely redistributable," and descendants of BSD -- led by FreeBSD,* but also including OpenBSD* and NetBSD* -- are vigorous participants in the contemporary battle for operating-system supremacy. Yahoo, arguably the world's busiest Web site, runs on FreeBSD. And yet, despite its proud heritage, BSD's current status doesn't quite match up to its early fame. A victim of schisms within its own developer community, bruised by the battle with AT&T and wounded by the defection of Joy to Sun, BSD is currently a small player, especially as compared with Linux. Linux-based operating systems have seized the public imagination.
BSD patriots argue that the battle is far from over, that BSD is technically superior and will therefore win in the end. That's for the future to determine. What's indisputable is BSD's contribution in the past. Even if, by 1975, Berkeley's Free Speech Movement was a relic belonging to a fast-fading generation, on the fourth floor of Evans Hall, where Joy shared an office, the free-software movement was just beginning.
The connection between the two movements is clear, if not direct. By demonstrating the power of cooperative software development, and by strengthening the software backbone of the Internet so it could further nurture such development, BSD helped enable the creation of a medium that will do more to spread free speech than anything hitherto constructed. Power to the people, from the code.
Many nice, upper-middle-class Berkeley backyards boast a redwood patio, possibly a hot tub, perhaps a vegetable garden complete with thriving rosemary bushes and marauding raccoons. Bob Fabry's backyard has a radio tower that wouldn't look out of place at a major Air Force base. Capable of telescoping upward to a height of 100 feet, and built mostly by Fabry himself -- the hub that rotates the antenna was scavenged from a 1940s airplane propeller -- the radio tower looms beside Fabry's home like a not-so-miniature Eiffel Tower. One look at it and you realize you are in the presence of a very dedicated geek.
Fabry takes his ham radio "hobby" seriously. He once even helped organize a trip to the uninhabited wilderness of Heard Island, about 1,000 miles north of Antarctica, just to set up a ham radio station for a few days so amateur-radio enthusiasts all over the world could enjoy the pleasure of exchanging radio signals with the faraway station.
But Fabry's most impressive achievement scales far beyond his tower or his expeditions. He is the Berkeley computer science professor who orchestrated the creation of Berkeley Unix. Not that he wrote a lot of code -- that honor belonged primarily to Joy and the other members of Fabry's Computer Science Research Group, an all-star band of programmers whose roster included names like Sam Leffler, Kirk McKusick, Mike Karels and Keith Bostic. But while Joy and others were hacking for 36 hours at a stretch, improving file systems,* networking performance, memory utilization and a hundred other arcane but crucial elements of Unix, Fabry was running interference -- maneuvering through the formidable bureaucracies of the University of California and AT&T, dealing with departmental politics and backbiting and, most important, writing the grant proposals that brought a steady flood of DARPA money into Berkeley.
Fabry was personally responsible for bringing Unix to Berkeley. His reasons were simple, and offer an early example of the pragmatist bent that has characterized BSD development ever since.
Unix was cheap. AT&T had been forced to practically give it away for free by government order. But Unix was also, fundamentally, a hack designed to work on cheap hardware. Back in 1969, Thompson wanted to get a computer game called Space Travel working on a castoff PDP-7. So what did he do? He wrote an entire operating system that made it possible. Kind of like using a nuclear missile to hammer a nail -- but then that's often standard operating procedure for obsessed hackers.
Fabry was entranced by Unix's affordability, along with the ease with which it could be adapted to different computer hardware. In addition to his academic research focus on operating systems, he was involved in setting up computing resources for the UC-Berkeley student body. In the mid-'70s, this could be an expensive proposition. Typically, operating systems that would allow multiple users of a mainframe* to work at individual terminals were designed only for extremely expensive computers. The costs per user could easily reach $50,000 a terminal, which made such systems impractical for pedagogic purposes. But Unix, used in combination with a relatively inexpensive PDP-11 from DEC, ended up costing closer to $5,000 "per seat."
Even better, a $99 license fee bought you access to the Unix source code -- to the blueprints, the magic recipe, the key that unlocked all hidden mysteries. For researchers, teachers and students, this was priceless. Researchers working on cutting-edge operating system technology could experiment with already existing source code and modify it for their needs; students who wanted to learn how an operating system really worked could find out by getting their hands dirty with the code. Duane Adams, the DARPA contract "monitor" who administered the Berkeley Unix contracts, notes that the availability of source code was an explicit reason why DARPA chose Berkeley Unix instead of contending aspirants like DEC's* VMS.* Never mind that VMS had been designed from the bottom up for the DEC VAX computers that were the most popular hardware for Arpanet nodes; VMS was a closed, proprietary system. You couldn't get in and muck about, so it just wasn't attractive to researchers.
DARPA was also recognizing reality. Prominent researchers, hungering for the magnetic tapes carrying Berkeley's latest distributions like so many desperate junkies, demanded that DARPA adopt Berkeley Unix because that's what they were already using.
"What was driving DARPA," says Fabry, "was that almost all of their contractors were telling them that they were running Berkeley Unix and it was superior to anything else available."
As one popular explanation has it, Unix's source code became widely available through a lucky accident -- as an unanticipated consequence of the consent decree that forbade AT&T from commercializing its non-telephony-related inventions. But that's only a part of the story. Unix was always more than just a bit player in a showdown between the world's largest government and the world's biggest corporation. Unix was, at least in the mind's eye of scientists like Fabry, "a thing of beauty." And from the very beginning, Unix benefited from a communal vibe that spread directly from its creators, Ritchie and Thompson.
Fabry recalls grasping the hidden wonders of Unix one week in 1975 when Thompson conducted a "reading" of Unix over several successive nights.
"The first meeting of the West Coast Unix User's Group had about 12 or 15 people," recalls Fabry, a mild man, now 60 years old, who clearly delights in his 25-year-old memories. "We all sat around in Cory Hall and Ken Thompson read code with us. We went through the kernel* line by line in a series of evening meetings; he just explained what everything did ... It was wonderful."
The reading of the code: Thompson's primeval act of deconstruction was an initiation into the Unix cabala, a ritual passing down of code lore. Fabry may have brought the first physical manifestation of Unix to Berkeley, but Thompson's reading embedded it in Berkeley's soul. Eric Allman, *" who was later to write sendmail,* the open-source-software mail transport program that still shuttles the vast majority of Internet mail across the Net, was an undergraduate at Berkeley when he attended the readings. He still has his marked-up "listings," reams of cheap, flimsy computer paper with notes scribbled on them, detailing the obscurities of the C programming language and other Unix arcana.
"The really bizarre thing is that Ken Thompson did a free tutorial on Unix kernel internals," recalls Allman, "and everyone fit into a rather tiny room." Today, you'd need to rent a ballroom.
Fabry marched against the Vietnam War while he was a graduate student in Chicago, and notes proudly that in his entire 12-year tenure at Berkeley he never once wore a tie. But although some historians have later described the Berkeley hackers as freedom fighters -- especially in the context of their battle with AT&T (which came well after Fabry had retired) -- neither Fabry nor the hackers themselves saw what they were doing in such explicit ideological terms. But when I ask Fabry if there was ever a moment when the goal crystallized in his head that software should be free, he turns the question around:
"Where did it come from that code should cost money? I think that's the fair question," says Fabry. In the mid-'70s, most programmers had grown up in an era where software was usually included with hardware and not considered a separate revenue source or proprietary intellectual property. Joy saw his work on Unix as research, to be shared with the rest of the academic research community the same way professors had been sharing the fruits of their labors for thousands of years. Ritchie and Thompson wanted their software to be used -- they did everything in their power to help the Berkeley programmers fix bugs and make improvements.
None of them saw himself as a crusader. But a trickle of idealism still occasionally leaked out.
"I think the spirit in which we were putting this all together was much the spirit that was picked up later by the Free Software Foundation and the various people who were trying to build 'software for the people,'" says Fabry. "The idea is that there is no duplication cost for software, so it ought to be basically free, and we were all working together to try to produce this ideal system that we would all love to have, and love to be able to use ourselves. That was the goal of a lot of people, and of course that was the original goal of Ken Thompson and Dennis Ritchie in starting Unix."
Fabry retired at age 43, tired of the DARPA treadmill and eager to focus his energy on his ham-radio tinkering. But Berkeley Unix's record of success still thrills him.
"Berkeley Unix was clearly the most successful university software project that has ever gone on," says Fabry in a rare moment of assertiveness, before backtracking slightly. "I don't know, I haven't been keeping up since 1983 and maybe there's been something since then, but I believe that that was true at the time. We had literally thousands and thousands of installations, and a whole generation of computer science students all around the world grew up on Berkeley Unix. It set a standard for operating systems that people are still having trouble doing better than. It was also the first efficient networking solution; for years it was the only game in town, the basis of Internet development. It really was one of the things that the people who made the Internet what it is today built on. There were battles that had been solved that didn't have to be solved again in order to do whatever new part that they wanted to do."
"Bill codes like a demon." -- John Gage
"His code was ugly." -- Kirk McKusick
"Bill Joy was a fabulous marketer." -- Eric Allman
"Bill was superb. He was the epitome of what one would like to see in a graduate student. -- Bob Fabry
"He had an infectious enthusiasm about him, where he would just get the people around him to do stuff. And he had an incredible drive to get his software out so that other people would use it." -- Kirk McKusick
"Berkeley Unix was the work of many talented developers. Bill Joy's particular genius was in integrating the work of these many contributors from many different organizations." -- Rob Gurwitz
"Bill's really smart." -- Sam Leffler
- - - - - - - - - - - - - - - - - - - - -
Gage's eyes twinkle as he recalls one of his favorite Bill Joy stories. The scene, he says, is in a boardroom high up in a building overlooking Washington, D.C. The time is the early 1980s. In attendance are some representatives from DARPA, some employees of BBN* (a Boston company that received the original DARPA contract to build the Arpanet) and a few Berkeley hackers, including Joy.
At issue was an annoying problem that had been bothering DARPA. DARPA had given Berkeley a major contract to enhance Unix so that it would be suitable for DARPA's network of research sites. It had also decided that Berkeley Unix should incorporate TCP/IP,* a specification for how Arpanet machines would interconnect. Devised by Vinton Cerf and Bob Kahn, TCP/IP (Transmission Control Protocol/Internet Protocol) was -- and still is today -- the basic method by which computers talk to each other across the Internet.
But DARPA had given BBN the contract to implement the TCP/IP protocol,* to write the all-important TCP/IP "stack."* Joy had been instructed to plug BBN's stack into Berkeley Unix. But Joy refused to do so. In his opinion, BBN's TCP/IP wasn't good enough. So he wrote his own high-performance TCP/IP stack.
As Gage tells it, "BBN had a big contract to implement TCP/IP, but their stuff didn't work, and Joy's grad student stuff worked. So they had this big meeting and this grad student in a T-shirt shows up, and they said, 'How did you do this?' And Bill said, 'It's very simple -- you read the protocol and write the code.'"
"That really frosted the BBN guys."
Well, sure. In programming lingo, a flat statement like "Read the protocol and write the code" is, to borrow some modern slang, a major dis. But did Joy really say the words? And did BBN's code really not work?
No and no, says Rob Gurwitz, the BBN programmer who wrote BBN's implementation of TCP/IP. Gurwitz says he was at all the DARPA steering committee meetings that handled TCP/IP matters during that era, and he doesn't remember ever hearing Joy make such a statement. Gurwitz, who says he worked closely with Joy and Sam Leffler on the integration of TCP/IP into Berkeley Unix, also says Joy's version of TCP/IP was not a direct replacement for BBN's code. Joy's stack was designed to maximize performance over local area networks that had wide bandwidth connectivity -- like an Ethernet* network designed to serve an entire university campus, for example. Gurwitz's version was built to operate on the much narrower 56Kbps telecommunication lines that made up the Arpanet's backbone.
Nonetheless, the Berkeley hackers were (and are) convinced that their implementation was superior, and they continued to resist all attempts by DARPA to force them to include the BBN version in Berkeley Unix, even though, according to Gurwitz, several DARPA sites continued to use BBN's version for years. As is the case with most programming disputes, the deeper one delves into the TCP/IP spat, the more "Rashomon"-like the search for truth becomes.
So why is the squabble important? For at least three reasons.
First, the incorporation of TCP/IP into Berkeley Unix can be, and often is, singled out as the most important innovation that made the Internet function efficiently. Joy and other Computer Science Research Group* veterans argue that their version of TCP/IP was crucial, because only it was technically good enough to satisfy researchers who wanted to communicate with each other and get work done on their local networks as well as on the Internet.
The TCP/IP stack, one could argue, was the original Promethean gift of fire to the mortals of the Net. And when the Internet suddenly boomed in the '90s, Berkeley Unix scaled up right along with it -- a testament, says Kirk McKusick, to the quality of its design. To this day, BSD advocates contend that the networking performance of BSD, which can still be traced all the way back to Joy's TCP/IP code, outclasses the best that Linux-based operating systems can do.
Second, the TCP/IP stack played a deciding role in settling the legal battle between AT&T and the University of California. The breakup of the AT&T monopoly in 1984 finally permitted AT&T to commercialize Unix. For years, as Unix became the preferred language of the Net and academia, AT&T had steadily increased licensing fees from the original $99 all the way up to $250,000. But AT&T wasn't the only interested party. In the early '90s, BSDi,* a spinoff of Berkeley's CSRG, started selling its own version of Berkeley Unix, and the University of California had been selling its version for years. In 1992 AT&T sued both the University of California and BSDi, claiming that BSD Unix included proprietary AT&T code.
Unfortunately for AT&T, the version of Unix that the company was then pushing, System 5, turned out to incorporate large chunks of code originally written by BSD hackers -- including the TCP/IP stack. Berkeley released all its code under an extraordinarily liberal license -- basically, users could do anything they wanted with BSD code as long as they retained the University of California copyright. But AT&T had stripped the UC copyrights and begun marketing the software as its own. Hackers like McKusick were peeved.
"We had written this code and they were claiming it was theirs," says McKusick, "and that they had the rights to it, and we just flat out didn't believe it. And it pissed us off that they were basically taking our work, that they hadn't paid a penny for, but that they had made money off of because of all the damn licenses that they had sold, and they were now trying to claim it was theirs."
The University of California's lawyers seized upon the opening, countersuing AT&T for copyright violation. After the requisite legal scurrying, the two sides came to a settlement, the terms of which both sides are forbidden to comment on. That settlement, along with a concerted effort by BSD hackers to rid their code of any AT&T "taint," freed the operating system of its last proprietary vestiges.
Third, even if Joy did not piss off his fellow programmers by saying "Read the protocol and write the code," no one who knows him well will deny that it is the kind of thing he could easily have said. Joy's colleagues and professors are unanimous in describing him as a fundamentally nice guy. But like so many great hackers, Joy is also almost unconsciously arrogant. And that arrogance has been, historically, a key part of the BSD legend. As a general rule, programmers tend to have a high opinion of themselves. And as a class, Unix programmers are well known for demonstrating their own special blend of high-priest orneriness. But BSD Unix hackers, with some notable exceptions, are especially virulent in their self-assuredness. They aren't wrong very often, and when they are, convincing them of that fact requires several armies and quite a bit of heavy artillery. Indeed, the easiest explanation for why BSD hackers watch in dismay while Linux-based operating systems sweep the world is that, for years, subsections of the BSD community have been endlessly imitating the mulishness that marked Joy's original reluctance to compromise on TCP/IP.
Of course, if anyone ever had a right to be arrogant, it would be Bill Joy. When the University of California received new computer terminals advanced enough to allow a cursor to be mapped to a particular point on the screen, Joy promptly, and speedily, wrote a text editor, vi,* that took advantage of the new capabilities. Vi is still widely used today, standard equipment on nearly all Unix installations.
If the compiler Joy was using didn't satisfy him, he wrote a new one. If the backspace key didn't work correctly in Unix, he rewrote the source code. And so on.
"Bill's very good at taking something," says McKusick, "saying, 'OK, this is what I have, this is where I want to get to, what's the shortest path from here to there?' His code was ugly, unmaintainable, incomprehensible, but by golly it only took him two weeks to do an incredible amount of functionality. Someone asked me once to compare myself to Bill Joy, and I said, "You know, there's really nothing that Bill's done that I couldn't have done, but what Bill did in a year would take me 10.'"
"He was just very good at reading a large body of code and wrapping his mind around it," recalls Fabry. "So he could do major reorganizations of code in a single weekend. I saw him do major reorganizations of the kernel several different times. In the beginning it took him just a few days, while later on it might take him a month. It was wonderful."
Joy called me once on his cellphone. It was Feb. 14, and he was in a grocery store in Monterey, Calif., stocking up on food before heading over to the annual TED (Technology, Entertainment, and Design) conference. Never one to waste a spare second, he decided to combine two chores -- my questions about his Berkeley days and his own need for sustenance. The result gave me a close glimpse at one of Joy's more famous qualities -- his ability to multitask. While answering a question from me, he would simultaneously talk to the cashier or a counter person without skipping a beat. In the middle of a sentence, out would pop the words "fruit salad" or "yogurt with raisins."
Like Unix itself, famous for its ability to perform simultaneous tasks, Joy could allocate portions of his brain to separate jobs at the same time, without appearing to shortchange any of them.
I learned later that his performance wasn't as awe-inspiring as I first thought. Joy, I was told, decides what he thinks about something and then rarely changes his mind; instead, he stores away his thoughts on the subject and is able to regurgitate them on demand, without wasting any fresh brain cells.
People who've worked with Joy say his ability to compartmentalize made him difficult to deal with on occasion. The stories of shouted arguments ringing through the hallways of Berkeley's computer science department are legion. But Joy's stubborn single-mindedness also made him an excellent leader. Berkeley Unix thrived in large part because Joy held his code to the highest possible standard and refused to compromise. Leffler, Joy's second-in-command for most of the heavy lifting involved in pushing out the first versions of Berkeley Unix, says he and Joy had a responsibility not to compromise. The U.S. Department of Defense was paying for BSD, and its prospective users encompassed the cream of the computing-science crop.
Intriguingly, Joy doesn't subscribe to the fundamental credo of the Linux movement -- the belief that the strength of open-source software is its ability to tap the energy and enthusiasm of a vast network of volunteer programmers. Linux is built on an egalitarian ethic: Perhaps not every programmer will write great code, but together, all those eyeballs and all those keyboard-pounding fingers will incrementally make their way toward greatness.
But Joy doesn't believe that having more programmers equals better code.
"Most people are bad programmers," says Joy. "The honest truth is that having a lot of people staring at the code does not find the really nasty bugs. The really nasty bugs are found by a couple of really smart people who just kill themselves. Most people looking at the code won't see anything ... You can't have thousands of people contributing and achieve a high standard."
Joy even disputes the commonly held conception that BSD set the model for demonstrating how a large software project could be built via contributions by a widely distributed network of programmers.
"Almost no fixes came in from anywhere else," says Joy, referring to Berkeley Unix. "In fact, most of the stuff I got back was not that great. Remember, there wasn't a real swift network at that time. In later years you got stuff back, but I didn't get much stuff back during that time that I was doing."
Joy may be overstating the case. Leffler and McKusick, while conceding that 90 percent of outside contributions did not meet Berkeley's standards, state authoritatively that significant portions of BSD code came from outside Berkeley. And after Joy's 1982 departure to Sun, the percentage of outside contributions began to rise, in tandem with the rise of the Net. Joy may be overreacting against the new generation of open-source hackers, many of whom frequently fail to acknowledge (or even be aware of) Joy's contributions to the software ecology that underlies the entire free-software movement. Joy says that when he boots* Red Hat Linux, he sees boot-up messages scroll by that he personally wrote 20 years earlier. Joy would much rather talk about his current passions, Sun's Java and Jini, and when he's asked about Linux, he sometimes lets traces of annoyance slip through. Who are these punk hackers, some of whom weren't even alive the first time Joy rewrote the Unix kernel?
"If I had to rewrite Unix from scratch, I could do it in a summer, easily," says Joy. "And it would be much better. A much, much better job. The ideas are old."
But if an increase in the number of programmers doesn't ensure an increase in the quality of code, then why do Linux-based operating systems dominate the market today? And if Joy rejected most contributions from outside Berkeley, why does BSD enjoy a reputation for pioneering the model for collaborative open-source software development?
"BSD was Bill Joy, initially," says McKusick. "He did the distributions and talked about them and pushed them out. He would give talks, and, inevitably, at the end of the talk he would say, 'And if you have any cool stuff, come talk to me.'"
But, Eric Allman hastens to interject, Joy did not invent the concept of freely redistributing software at Berkeley. That "seemed to be an ethos that permeated Berkeley in general," recalls Allman, who had worked in the early '70s on a database project called INGRES that was widely redistributed. And after Joy was gone, that ethos, says Allman, continued.
It is a gorgeous spring Sunday in March. McKusick, Allman and I are sitting around an outdoor table in the backyard of "Chez Oxford," the north Berkeley cottage that Allman and McKusick have lived in for nearly 20 years. The first warm sun after a month of nearly continual rain is beaming down upon us, combining happily with a stream of selections from Chez Oxford's copious wine cellar.
Allman and McKusick can use some relaxing. Allman's company, Sendmail, is in the midst of a hectic round of financing and has just released a major upgrade. McKusick, meanwhile, has sent the entire BSD community into a tizzy by orchestrating a merger between BSDi, a spinoff from the CSRG that sells a proprietary version of BSD, and Walnut Creek CD-ROM, the largest distributor of the FreeBSD distribution. Plans are even afoot for the migration of some code from BSDi's proprietary BSD/OS into the completely free FreeBSD. Interpretations of the merger's significance vary wildly. To some, it's a concession that BSDi's proprietary offerings are making no headway against the Linux onslaught. To others, the merger is a hopeful sign that the days of BSD splintering are over: The community is re-forming again, readying itself for a new round of sparring with other operating-system upstarts.
If Joy was the heart of BSD, then McKusick is its soul, the keeper of the Berkeley Unix flame. After Joy's 1982 departure, Leffler lingered around Berkeley long enough to complete the delivery of BSD 4.2 to DARPA, and then also left academia for the more lucrative embrace of Lucasfilm and, later, Silicon Graphics. McKusick, who had until then been juggling his dissertation with various BSD tasks that Joy talked him into doing, picked up the reins.
To this day, McKusick says proudly, he is one of the only original BSD developers who subscribes to developer mailing lists for all the major BSD spinoff distributions. Every night at 2 a.m., one of his computers downloads that day's modifications of FreeBSD and recompiles the system, keeping him excruciatingly up to date. If anyone can claim a comprehensive perspective on BSD's evolution over time, it is McKusick.
Like his fellow BSD coders, McKusick disavows revolutionary fervor or Berkeley radical ambitions.
"The contribution that we made, ultimately," says McKusick, "was in developing a model for doing open-source software ... We figured out how you could take a small group of people and coordinate a software project where you have several hundred people working on it."
McKusick outlines an organizational model that grew up in the wake of the departure of Joy and Leffler. At the center, there is a "core group" -- a set of programmers who control access to the code, by granting or revoking the right to modify or "commit" new code to the code base. Spreading out from them are the "committers" who have that right. Extending out beyond the committers are the general community of developers who submit changes, bug reports and fixes to the committers. Most of today's high-profile open-source projects, such as the Apache Web server and the GNU project, use a similar form of organization. Linux is the major exception. There is no core for Linux -- just Linus Torvalds, followed by a tier of trusted "lieutenants."
"The committers," says McKusick, "were a group of people we trusted to commit stuff that were responsible for things. The notion was that you didn't have all these autocratic controls ... Now, you could have snuck in and committed something to the kernel, some kind of trapdoor even. I won't say we wouldn't have been none the wiser, because we did in fact keep close tabs on what was going on in the kernel, but we didn't need to tell people not to do that; we didn't have to administratively keep them from doing things they shouldn't be doing. We had set up a culture as well as a structure."
Still, 90 percent of the contributions were thrown away; the rest, as McKusick likes to say, "were peed upon to make them smell like Berkeley.
"The trick is that ultimately you find that small nugget of people who are the really good ones, and that's why we have this whole hierarchy, that's why we still have a hierarchy today," says McKusick. "Yes, there are thousands of developers, most of whom honestly couldn't paint themselves out of a wet paper bag if their life depended on it ... But you still give them a place; you don't just dismiss them."
The circle has widened constantly, McKusick notes. Today, FreeBSD has a core of 16, surrounded by nearly 180 committers and thousands of developers. Even so, FreeBSD is dwarfed by the development community contributing to Linux-based operating systems. Which raises the obvious question, one that McKusick has heard hundreds of times over in recent years: How did Linux-based operating systems overtake BSD?
There are some obvious answers. In the early '90s, as the power of personal computers grew steadily, many Unix aficionados began seeking a way to run Unix on their PCs. There were two contenders at the time, 386BSD, a version of BSD created by William and Lynne Jolitz for computers built around Intel "x86" microchips, and Linux-based operating systems. But the AT&T suit, combined with the slow pace of development on 386BSD, placed the whole BSD effort under a cloud. No one knew if AT&T would succeed in quashing BSD altogether. Linux, in combination with the GNU utilities, was protected by the ironclad GNU General Public License -- all the code was free and always would be free.
By the time the AT&T suit was resolved, the snowball ride to Linux was underway. And the future development of BSD after 386BSD did little to persuade hackers to change their minds. Developers dissatisfied with the pace at which 386BSD incorporated new patches split off and founded FreeBSD and NetBSD in 1993. Not long after, an internal dispute within the NetBSD core resulted in the spawning of OpenBSD. Meanwhile, McKusick and other members of the CSRG founded BSDi.
McKusick shrugs off the widely held perception that the BSD community is irreparably shattered. In addition to the just-merged FreeBSD and BSDi, he declares, there are only two other major distributions of BSD, each of which has its own particular focus. NetBSD specializes in porting BSD to different computer architectures. OpenBSD concentrates on security issues.
And how many Linux distributions are there? asks McKusick, rolling his eyes. Fifteen, 20?
I point out that all the Linux distributions share the same kernel, which is overseen by the strong, and generally impartial, centralizing presence of Torvalds. BSD has no center.
McKusick looks a little wistful.
"This is somewhat egotistical," he says. "But I believe that had I been willing to act as the figurehead ... I could have kept the community from splitting. I had basically been in that position for 10 years, maybe 15, and I had really felt that the time had come to pass the mantle on to other people. I had a certain view of the way things ought to be done, and at some point you want to get new blood. Honestly, what I really thought was going to happen was, I knew that things would explode and there would be a lot of different [distributions], and I figured most of them would die off, and one would become the evident new one. And in some ways that really has happened. I mean FreeBSD has, at least by seats, 80 percent of the market, and the other two are interesting, but they are really out in the noise."
Depending too much on a centrally unifying force, suggests McKusick, isn't necessarily a strength. "What happens when Linus Torvalds either dies, gets tired or otherwise steps away from it all? It's a huge weight. He's done, in my mind, a terrible job of building something that lives beyond him. If I got hit by a bus, BSD wouldn't really be affected a lot."
As proof of his assertion, he notes that Torvalds does not rely on a source code control software* program to administer changes to the Linux kernel. Instead, Torvalds reviews each major patch by plugging it into his own development system. But what happens if something goes wrong? How do you roll back to before the changes? Where is the institutional memory, as embodied by software, that will keep a project going when the original leader leaves?
When I e-mailed Torvalds after this discussion for a response, he dismissed the problem.
"The purely technical side of keeping track of the sources can be handled by source control packages," says Torvalds, "but at least, in my opinion, they actually tend to favor the approach of 'Let's put this in now; if it turns out to be a mistake, we can always revert it because we have source control.' And of course, nobody ever actually does clean up anything. Or hardly ever. So I think the real problem in computer science is to have quality control before it even hits the distribution, and so far there isn't any other package than the human brain that can do that job."
Torvalds' answer is interesting, if only because the confidence that it reveals echoes the strength of Joy's belief in his own abilities. Perhaps Linux will not be able to develop BSD-like organizational structures until after Torvalds leaves the scene.
Ultimately, the disagreements over organizational strategy between the BSD and Linux camps are petty when compared with the similarities. As McKusick is quick to stress, "The important thing is to be open source, not whether Linux or BSD wins out."
And in that context, one can indeed see BSD, from the Joy era through the McKusick era and beyond, as part of a continuum feeding energy and enthusiasm into the current Linux upswing. Even during the earliest days, Joy, as McKusick notes, wanted other people to contribute "cool things." Ever since then, the trend line has been one in which the circle of contributors has widened steadily. BSD demonstrated how to bring people into the process -- by making them part of the process and by giving them credit for their contributions. As Keith Bostic,*, the leader of the drive to create a version of BSD that included no proprietary AT&T code at all, likes to say, "There is no end to the world of good you can do by giving people credit."
Because what happens? You end up creating tools that everyone can use to be even more productive, to create even-greater structures that encourage collaboration, which in turn unlock even more creativity. That's the essence of free software, and it goes beyond what kind of license protects the software, or whether source control software is employed, or even how arrogant the top developers might be. By opening up the code, Berkeley widened the pool of software possibility.
- - - - - - - - - - - - - - - - - - - - -
Gage* didn't contribute to the code in Berkeley Unix, but he spent plenty of time hanging around the computer rooms in Evans Hall, gabbing with Joy and pondering the political implications of the computer revolution. Gage, a mathematical statistics graduate student at Berkeley, was also a hippie radical from way back -- involved in both the Free Speech Movement and the antiwar protests. He was a delegate for Bobby Kennedy at the Democratic Convention in 1968 and deputy press secretary for presidential candidate George McGovern in 1972.
After three hours in Gage's favorite Berkeley coffee shop exploring what the Internet means for individual liberty and what role Berkeley Unix played in catalyzing the Internet's growth, Gage rolls his memories back to perhaps the single most-famous moment in Berkeley's history of activism.
He starts quoting the speech Mario Savio gave on the steps of the administration building overlooking Sproul Plaza. He hunches over the table, his eyes blazing with a sudden visceral intensity:
"'There is a time when the operation of the machine becomes so odious,'" declaims Gage, "'makes you so sick at heart, that you can't take part; you can't even passively take part, and you've got to put your bodies upon the gears and upon the wheels, upon the levers, upon all the apparatus, and you've got to make it stop. And you've got to indicate to the people who run it, to the people who own it, that unless you're free, the machine will be prevented from working at all!'"
Gage grins. Berkeley Unix, he proposes, offered a different way forward from the painful agony of hurling oneself into the operation of a demonic crankshaft. Berkeley Unix, with its source code available to all who wanted it, was the "gears and levers" of the machine. By promoting access to the source code, to the inner workings of that machine, the free-software/open-source movement empowered people to place their hands on the gears and levers, to take control of their computers, their Internet, their entire technological infrastructure.
"The open-source movement is a free speech movement," says Gage. "Source code looks like poetry, but it's also a machine -- words that do. Unix opens up the discourse in the machinery because the words in Unix literally cause action, and those actions will cause other actions."
Savio is dead. The Free Speech Movement is half-forgotten. Few, if any, of its participants would have predicted at the time that a network of computers might prove to be free speech's greatest friend and best weapon. Indeed, Savio's "machine" was in part a metaphor for what he saw as the dehumanization inherent in information technology: The University of California was IBM, the students were punch cards, both literally and figuratively, fed into the machine, not to be folded, spindled or mutilated.
The Berkeley Unix hackers, by helping to unleash the power of the Internet, rehumanized the "machine." Those "words that do" instigated connectivity and provoked communication. Somewhere, Savio is smiling.
Shares