Red Hat — a company that also plans to generate revenue from selling Linux support services — did not appreciate the humor inherent in its trademark being emblazoned on a risque poster advertising the services of a direct competitor. So Red Hat summoned its lawyers. The posters had to come down.
Lawyers? Cease-and-desist threats? In the cozy world of Linux, where every other hacker takes joy in proclaiming membership in the “community” and promises that the nasty old rules of greedy capitalism will soon be rewritten? How could this be?
Yes, it’s been clear for at least a year that the Linux start-up scene is fast maturing into a serious commercial battleground. But one of the elements fueling that growth is the widespread belief that the rise of Linux — and all open-source software — offers something different from the same old fangs-bared, cutthroat business bullshit.
On the bulletin boards and mailing lists where every Red Hat move is dissected in voluminous detail, some critics suggested that Red Hat’s heavy-handed behavior was yet one more sign that Red Hat is evolving into the Microsoft of Linux. It’s not the first time such charges — the worst invective that the free-software world can spew — have been leveled at Red Hat. But in most cases, the charges have little merit: They’re usually spouted by hormonally turbo-charged juvenile delinquents looking to stir up some flames.
Now, however, the Linux marketplace is exploding. Red Hat has already filed for a public stock offering, and other Linux companies are sure to follow. Two competing Linux distribution vendors, TurboLinux and S.u.S.E., are aggressively reaching out from their home bases in Asia and Europe into the North American market. Newcomers like LinuxCare, fueled by massive venture capital investment, have rocketed onto the scene. The symptoms of hypergrowth are everywhere: Companies are staffing up, stuffing new hires in every available office nook and cranny, and struggling madly to manage the pain of pell-mell expansion.
Pity the market leader in such a climate. Red Hat spokeswoman Melissa London laughs at the Microsoft comparison:
“It’s so funny — when was the last time you saw Microsoft make its operating system available for free download and remain committed to that? Our commitment to that is going to speak for itself. We take a fair amount of shots because we are in a leadership position, but this is too exciting a position to muck it up with Machiavellian initiatives.”
Still, now more than ever before, Red Hat is facing pointed scrutiny, and not just from loudmouth programmers trolling online discussion forums. Red Hat’s competitors are also asking questions. Does Red Hat support efforts to create a common standard for Linux? Or does its high-profile employment of top “inner circle” Linux hackers give the company an unfair advantage in determining just what that standard is? Is Red Hat purposely making it hard for other distribution vendors to keep up? Is it positioning itself to be the single dominant player in a market that prides itself on its wild diversity?
Judging from a close look at the evidence, the answers to these questions are mixed: Yes, Red Hat is striving as best it can to gain a competitive advantage, but no, it is no Microsoft. And it may well never be — as long as the code stays free.
But there’s also no question that the Linux kids are growing up. The poster brouhaha may well have been just a silly spat, but its symbolism is potent. The stakes in the Linux game are rising, and so is the friction.
A year ago, no one had ever heard of LinuxCare — the company did not exist. But co-founders Art Tyde and Dave Sifry are hardly newcomers to the world of Linux. Tyde founded the Bay Area Linux Users Group in 1994, and met Sifry at the first meeting. The two men are peas in a pod — quick to laughter, equally facile with the well-polished sound bite and boasting long records as successful computer programming consultants. Listening to them smoothly finishing each other’s sentences as they talk about the “empowering” aspects of open-source software, you can easily imagine them knocking the socks off potential investors — which may explain why LinuxCare is the only Linux company so far to have attracted the direct backing of Silicon Valley’s premier venture capital firm, Kleiner Perkins Caufield & Byers.
Today, veterans of the Linux scene express astonishment at just how fast LinuxCare moved onto center stage in the Linux marketplace. But that’s what happens when “KP” gets behind you. Unlike other Linux startups, LinuxCare has enjoyed the luxury of having loads of cash to spend from the get go.
In February, LinuxCare announced that it had been chosen by the Dell Computer Corporation to provide support for Dell workstations pre-installed with Linux. Then, in March, LinuxCare made many friends by sponsoring a huge party at the LinuxWorld conference in San Jose, providing free beer, wine and food for thousands of attendees. The company also drew attention by placing a full-page ad in the Northern California edition of the Wall Street Journal nominating the creator of Linux, Linus Torvalds, for president of the United States.
The selection of LinuxCare to provide support for Dell computers that are running Red Hat’s distribution of Linux could be construed as a slap in the face to Red Hat: After all, a key leg of Red Hat’s business plan is to provide support for its own software.
“They chose us,” says Art Tyde, “because, quite frankly, we are better at it.”
Perhaps so (Dell was unable to comment by the time this story was posted). But one Linux insider says that the terms of the deal were so favorable to Dell that the decision was a no-brainer — that LinuxCare volunteered to provide support at extremely low cost as a canny marketing move.
Whatever the case, LinuxCare is clearly aiming directly at a market that other, more established companies have hoped to make their own. Which is one reason that eyebrows in the Linux community rose when Tyde was quoted June 10 in the Investors Business Daily as saying, “Red Hat is not LSB-compliant — most other distributors are.”
LSB is the Linux Standards Base, a fledgling organization attempting to define a common standard for Linux distributions. The worst fear in the Linux world is that the multiple Linux distributions will lead to the fragmentation of Linux — thus hurting its chances to continue to grow and prosper. A fragmented Linux would be a huge headache for support providers in particular, forcing them to be conversant in the different nitty-gritty details of an endless number of distributions.
Dave Sifry, president of LinuxCare, says Tyde was misquoted. As both he and Dan Quinlan, the chairman of the LSB, are quick to note, there is no standard yet for Linux specified by the LSB, so it is impossible to say whether Red Hat is LSB-compliant. Red Hat’s official statement on the subject is as follows: “Red Hat is extremely supportive of the LSB effort, and we think its success is important for Linux to succeed in any meaningful way. We’re actively participating in the LSB specification committee, and we’re going to make every effort to ensure that Red Hat tracks the LSB standards as they are finalized.”
Many Linux experts argue that Linux is in no danger of fragmenting, and suggest that the recurrent surfacing of the issue is just more FUD — “fear, uncertainty and doubt” — spread by Linux’s competitors. First of all, they note, all distributions share the same kernel — the heart of code that is the engine driving Linux-based operating systems. Second, nearly all of the code layered over that kernel is open source — freely accessible to the public. If your software application doesn’t work on a particular distribution, you can always look at the code, figure out exactly what went wrong and fix it.
But there are still some areas where fragmentation is a real concern — most centrally, in the “C libraries.” The C libraries are not a part of the kernel itself, but contain code that is used and reused by nearly all software applications. The original free software C libraries were developed by the Free Software Foundation founded by Richard Stallman. Today, ongoing work on the C libraries is carried out by a loosely linked group of programmers, most of whom work for various open-source companies.
Red Hat has a reputation for releasing distributions with extremely new versions of this critical code well before anyone else. This, say some of Red Hat’s competitors, can cause major headaches for the other distributions and software application vendors. Quite frequently, applications that worked on older versions of Linux don’t work on the new version.
“[With regard to] some of the core C libraries,” says Cliff Miller, CEO of TurboLinux, “there should be a common ground that all of the distributions share.”
Miller says that application developers ideally want their programs to be able to run on all the different Linux distributions, but that constant change in the C libraries makes it difficult for the developers to keep abreast. If there were a commonly accepted standard, says Miller, there wouldn’t be a problem.
“I think you’ll find agreement on that from most of the hardware vendors and almost all the application developers and the Linux distributions companies,” says Miller. “Except that Red Hat is much less hot on the idea. If you are the market leader, then you want to create standards — that’s probably a pretty natural feeling.”
TurboLinux, formerly known as Pacific HiTech, has a particular problem with constant change in the C libraries. Founded by the multilingual Miller in 1993 while he was a graduate student in computer science at the University of Utah, TurboLinux has long specialized in creating versions of Linux for the Asian market. Although Miller is now looking to make a dent in the U.S. market with TurboLinux’s premier product — a high-end Linux Web server that retails for $1,000 — Asia, especially Japan and China, is still TurboLinux’s bread and butter.
And that means dealing with the problem of Chinese and Japanese ideograms, or “characters.” Rendering such characters on a computer screen requires a specific technical approach that is often not “natively” supported by the standard C libraries. Special patches have to be written to fix the problem — and they must be rewritten for each new version of the C libraries. As Red Hat keeps releasing its own new distributions, TurboLinux has to scramble to keep up.
But is Red Hat pushing for the adoption of newer and newer versions of the C libraries out of some nefarious plot to keep the rest of the Linux world off-balance? Or is the company simply over-eager to move Linux technology forward?
According to Stephen Tweedie, a Linux kernel hacker employed by Red Hat, there is no plot.
“I have been working on Linux since the early days, long before Red Hat existed,” says Tweedie, who is careful to note that he is speaking on his own behalf as a Linux kernel hacker, and not as a spokesman for Red Hat. “My first experience with a Red Hat distribution came when I needed to do a major upgrade of the libraries on my home machine. Red Hat were the first to come up with a distribution based on the new libraries.”
“So from a user’s point of view, in my case that rapid deployment of the new technologies was a positive blessing,” says Tweedie. “To say that you don’t want to use the new versions of the [C libraries] is to hide your head in the sand — the libraries, like the rest of the Linux technology, are not developed by any Linux companies, and the community moves forward at different speeds.”
The LSB’s Dan Quinlan says that once the LSB finishes its specification for a standard Linux distribution, library incompatibilities should become less of a problem, because the specification will require that a particular configuration always be supported, and both application developers and distribution vendors will be able to depend on that. As for right now, Quinlan also doesn’t see any evidence of serious misbehavior on the part of Red Hat.
“If Red Hat did anything really bad there would be a huge backlash against them,” says Quinlan. “There’s no evidence of such a backlash. I can’t think of anything bad that they have done. They are definitely the biggest player in the Linux community now, and there’s a potential for a big company stepping on a little company, but that doesn’t mean they are being malicious. If all the kernel hackers at Red Hat resigned, then that would signal that something bad was happening.”
At present, there seems to be little indication that Red Hat’s kernel hackers are inclined to leave the company — but that fact itself is another major concern for some of Red Hat’s competitors. They are worried that Red Hat is stockpiling key Linux talent and thus gaining an advantage that could solidify its position as the dominant distribution vendor.
There’s little debate that Red Hat employs some of the most famous names in Linux, excluding Torvalds himself. Although every Linux company with money to spend is aggressively hiring key open-source programmers, no other company can boast as big a concentration of Torvalds’ so-called “top lieutenants.” Alan Cox, David Miller, Steven Tweedie and Doug Ledford are among the most highly respected Linux kernel hackers you could name. Torvalds may make the ultimate kernel decisions — ruling on questions such as what features will be included in a particular version, or when that particular version is considered finished — but he also trusts programmers in the tier right below him to make their own decisions.
One of the reasons that Red Hat has been able to attract these programmers is by guaranteeing that all Red Hat additions to Linux are protected by the “GPL” — the license that ensures that free software remains permanently accessible to all. But just making sure that code is free may not be enough, say some critics.
Lonn Johnston, vice president of marketing for TurboLinux, says that a representative of one “major hardware vendor” worried that Red Hat’s dominance in the market could lead to the vendor getting “Microsoft-ed.”
“This hardware vendor’s argument,” says Johnston, “was that if one distribution is successful in acquiring a disproportionate number of lieutenants … then that talent pool is not only driving the development of the operating system, but is also advising Intel [and other hardware manufacturers] what to do with its future hardware direction and development. So that even if the source code that they produce is GPLed on release, you don’t really know where they are going with it — unless you are sitting in on those design meetings or development meetings.”
In other words, Red Hat’s market dominance, which is in large part predicated on its employment of famous hackers, leads naturally to access to information about hardware that other companies do not have. Red Hat can then tailor its code to work best on that hardware. Even if all the code is released to the general public, Red Hat is still the first on the block to be able to take advantage of it.
This is a real advantage — but Red Hat isn’t the only company playing this game. VA Linux Systems, which also boasts a sterling array of well-known Linux hackers on its staff, has also been leveraging its market position to give it access to proprietary hardware information. “I have the same strategy,” says CEO Larry Augustin. “We do our best to get early access to chip design and other programming information, and we can then do the best Linux support for that. Part of the way we earn money is that that makes us first to market. We give away all the code, but we gain a technical support advantage. As the original author you can support it better.”
“Red Hat does have the ability to get access to such information under NDA [non-disclosure agreement] in some cases,” says Tweedie. “However, there is a very strict rule: Such NDAs can cover the development period only. As soon as the hardware and drivers are publicly released, all of the controlling source code must be opened completely — without any intellectual property caveats — to the public.”
“Yes, some players in the commercial Linux world might get to certain technologies or drivers faster than others if they are actively contributing to their development,” says Tweedie. “And yes, that means that other companies may feel that they are getting left behind if they don’t make a positive decision to contribute to Linux themselves in the same way. However, I cannot see that it hurts Linux as a whole in any way whatsoever as long as the end results of all of these efforts are all being contributed back into the same mixing pool. Furthermore, I can’t see any way to avoid this sort of thing if we are to be taken seriously by the industry.”
Ultimately, it all comes down to the code.
“That’s what I hang my hat on,” says VA’s Augustin. “If the code is GPLed it doesn’t matter — they [the Linux companies] can snipe at each other all they want — the code is there. It will always be there, we can’t take that away.”
And that, in a nutshell, is why Red Hat is where it is, says Sam Ockman, CEO of Linux hardware vendor Penguin Computing. “Red Hat has a better record of GPLing all its code than any other commercial distribution … Red Hat has always been the most free, the most GPLed, the most community supportive.”
Should Red Hat ever make a serious misstep, says Ockman, the hackers will desert the company and some other distribution will pick up the mantle — as well as all the code that Red Hat’s hackers ever wrote. So even if an over-ambitious CEO goes overboard in seeking that extra competitive edge that could set his or her company sliding down the slippery slope toward Microsoftish no-holds-barred total market domination, that CEO may be unlikely to succeed.
Sure, the friction between individual companies will no doubt continue, and the sniping may reach hitherto unimagined intensity as the dollar amounts involved continue to rise; but again, that’s only natural, and nothing to be afraid of. There’s always the safety of free software — the first, last and best weapon preventing any company in the Linux world from permanently calling the tune to which everyone else dances.