Ed Frauenheim

Crafting the free-software future

At VA Linux's SourceForge, thousands of programmers are collaborating for both love and money.

  • more
    • All Share Services

Crafting the free-software future

In between his two to three hours of homework every night, 16-year-old Julian Missig plays the part of a software project manager at SourceForge.net, a Web site-cum-watering hole for programmers looking for a place to hack. At SourceForge, in collaboration with hackers from all over the globe — Germany, France, Russia, the Ukraine — the New Jersey high school senior works on a program called Gabber.

Gabber is an offshoot of Jabber, an open-source instant messaging system — specifically designed for Linux-based operating systems that use the GNOME desktop environment. Only about 5,000 people are currently using Gabber, but that’s not what makes Missig’s work interesting. Instead, it’s the place he’s chosen for his programming.

Gabber is just one of roughly 16,000 software projects hosted by SourceForge. The only thing the software programs in development at SourceForge have in common is that they are all free, or open-source: The underlying code to the programs is made freely available to the general public.

Since launching in January 2000, SourceForge, which is sponsored by open-source hardware and services provider VA Linux, has attracted 125,000 software developers, say company representatives. The site added more than 17,000 software developers just between Jan. 18 and Feb. 21 and has been averaging monthly growth of more than 30 percent. SourceForge, says John T. Hall, vice president for strategic planning at VA Linux, is close to eclipsing IBM as the world’s largest community of software programmers.

But size isn’t the only thing that matters about SourceForge. To some observers, the community of developers that has gathered together at SourceForge represents nothing less than the latest stage of the evolution of free software. It’s a focal point for all the world-wide energy flowing out of hackers and into code that everyone can share. Eric Raymond, co-founder of the Open Source Initiative, a prominent advocate for open-source software and a member of the VA Linux board of directors, says the site is as significant to the open-source movement as the creation of Linux — which focused the efforts of thousands of volunteer computer coders worldwide.

“(Linux) created a kind of social synergy,” Raymond says. “SourceForge is having the same effect.”

Even so, Richard Stallman, founder of the Free Software Foundation, believes that SourceForge’s emphasis on the pragmatic open-source approach to software development distracts programmers from the altruistic principles at the heart of free software. But perhaps of even more importance is the question of how long SourceForge can stay afloat. Brian Behlendorf, a co-founder of the open-source Apache Web server software effort, and the chief technical officer of Collab.net, wonders what happens if VA Linux runs out of money. On Feb. 20, after posting a quarterly loss of $74.1 million, VA Linux announced it was slashing 25 percent of its 556-employee work force.

But the company recently invested $500,000 in new hardware for the site and has dozens of employees focused on SourceForge. And it spared SourceForge from the cuts — instead, executives called the Web site a priority and promised to beef up the site with additional hardware and staffing. Though VA Linux executives like to describe SourceForge as their “gift” to the free software/open-source world, they are also hoping that it will be a desperately needed key to commercializing open-source software development.

SourceForge is the answer to a free-software developer’s most basic need: computer resources for working together online. Programmers have been coding together for decades via electronic networks. The most famous example is what’s commonly called the Linux or GNU/Linux operating system. Linus Torvalds could never have orchestrated the creation of Linux without the help of the Internet, e-mail and Usenet newsgroups.

But creating sophisticated collaborative software projects online has never been easy. The biggest challenges include finding a Web server computer to host projects, managing the various versions of the code, administering the power to choose who can tinker with the software and keeping a handle on which bugs have been fixed.

VA Linux has a long record of hosting online collaborative workplaces. For years, one computer in a back room served as an important repository for work on the Debian GNU/Linux operating system. “(VA Linux) is based on the success of this community,” Hall says. “We thought we should give something back.”

By the summer of 1999, one of the company’s employees did nothing but handle the system administration duties related to project hosting. VA Linux executives, looking forward, realized that the free software world could benefit from a more automated approach to software development. And so was born SourceForge.

SourceForge’s mandate is to provide a solid infrastructure for the international, decentralized collaboration that has been at the heart of the free-software movement for decades. Services freely available at SourceForge include mailing lists, discussion forums, project Web pages, bug-tracking programs and a method for letting program leaders give selective permission to other hackers to make changes in the source code. SourceForge also enables managers to categorize their projects by programming language, software license type and intended audience. Project leaders also can post news items about their coding efforts.

Although some developers grumble at a clumsy user-interface, bandwidth shortages, and occasional system crashes, developers also appreciate what VA Linux has given them for free. One grateful user is Dan Kuykendall, a leader of the phpGroupWare project, which is building a program for business collaboration with features like e-mail, a calendar and to-do lists similar to those found in the commercial software Lotus Notes.

“It’s run so smoothly because of SourceForge,” says Kuykendall, 26. “[Without SourceForge] I wouldn’t be able to spend as much time on the code itself. I’d be managing the project.”

SourceForge, says Kuykendall, has made it easier to handle the tricky business of virtually leading a team of roughly 20 programmers. For example, the CVS system — a program for allowing developers to submit code, track changes and keep different versions straight — and bug-tracking programs allow Kuykendall to parse out work to project developers based on their preferences. SourceForge also provides a way to invite contributions from and evaluate newcomers who haven’t been given phpGroupWare programming privileges. Anyone reviewing phpGroupWare’s source code can submit a bug report and also propose a software fix.

“We’ve gotten a lot of developers that way,” Kuykendall says. “If they can prove that they understand our code — that they’re following our design of it — then we can invite them in.” For phpGroupWare co-founder Joe Engo, SourceForge has allowed him to make a mark in the free- and open-source software world. “I’m 100 percent proud of this effort,” says the 23-year-old coder. “When I first started this project, I never dreamed it’d get to this scale.”

Software developers are coming to see posting code at the site as a rite of passage, says Raymond. And by gathering developers in one place, SourceForge simultaneously helps them become more effective in their coding collaboration and more unified in their beliefs, Raymond says.

“Watering holes can be very catalytic places,” he says.

In Raymond’s view, a number of events have galvanized the open- and free-software communities, including the creation of the Free Software Foundation in the early 1980s, the development of Linux, and the launch of the open-source movement in 1998. “(With each of those developments, the community) became more close-knit, conscious and effective,” Raymond says. “SourceForge is another step in that progression.”

Brian Behlendorf isn’t so sure. While loath to criticize a generous effort by VA Linux, he’s concerned that SourceForge may foster fragmentation of the open-source community, because it may be easier to start a new project than help a related one in progress. Behlendorf also doubts the massive numbers of developers at SourceForge feel a strong connection to each other.

“My gut tells me that people don’t like to be part of a 100,000-person community. They like to be part of a dozen-person or 100-person community,” he says. “Scale isn’t everything.”

The sheer number of developers who’ve been drawn to SourceForge has surprised even VA Linux officials. They say they just expected to attract and host between 15 to 20 programming projects. Having 16,000 has forced them to reevaluate the whole project. The company won’t reveal precise costs associated with SourceForge, but one spokesperson noted that the site consumes bandwidth equal to 60 T-1 lines. Dozens of server computers have been deployed to house the various software projects hosted at the site, and VA Linux has a staff of seven people devoted to site upkeep as well as dozens of programmers focused on writing code that will by used both at SourceForge.net and the commercial SourceForge product, SourceForge OnSite.

VA Linux is betting that the software and methodology of SourceForge itself will be attractive to tech companies hoping to improve their engineers’ collaborative efforts. Historically, the bulk of programmers have worked largely in isolation from one another on fragmented sections of projects. The market success and stability of open-source software such as Apache, the various tools and utilities produced by the GNU project and Linux has made information technology firms eager to adopt the open-source approaches of tight teamwork and massive peer review, where thousands of interested hackers find and fix coding flaws.

SourceForge OnSite is a package that includes hardware, software and services that can operate behind a company’s Internet firewall for open and non-open-source programming. Essentially, it’s a distillation of the tools available for free at SourceForge. Already, one Silicon Valley powerhouse, Agilent — formerly the testing equipment arm of Hewlett Packard — is a customer. VA Linux won’t reveal the names of its other SourceForge OnSite customers. But Hall says there’s been “tremendous interest” from software companies.

“(SourceForge OnSite is) going to be a major part of our business,” he says. Which may explain why the seemingly altruistic SourceForge is surviving the VA Linux cuts. The day after the grim Feb. 20 financial report, VA Linux sent a memo to the SourceForge community explaining that SourceForge pulls in revenue through banner ads, corporate sponsorships and SourceForge OnSite.

VA Linux is not alone in attempting to capitalize on the process of creating open-source software. Brian Behlendorf’s Collab.net is aimed squarely at the same target. Collab.net has set up sites for about a dozen firms hoping to tap into open-source expertise and interest — one example is a site where Sun Microsystems invites coders to work on its package of personal productivity programs, StarOffice. A related effort launched by Behlendorf is Tigris.org, which is hosting online development of a number of specific open-source coding tools, such as a new and improved CVS program. Tigris.org has a total of 60 users. Both Collab.net and another starup, OpenAvenue, sell products designed to help information technology companies get their employees using open source-like collaboration tactics. Both these services run completely over the Web without a hardware purchase.

Behlendorf expects this type of product — which can be configured to include contributions from software engineers at business partners — to make up the bulk of Collab.net’s business.

There also are what might be called coding marketplaces. SourceXchange, part of Collab.net, links developers with open-source projects funded by businesses. Asynchrony.com hosts open-source and closed-source projects for developers and then markets resulting programs. Up to 90 percent of revenues go back to the programmers — “Code for love and money!” the site urges.

Behlendorf and Hall are both bullish on the prospects of open-source tools and methods becoming a valuable, potent force in the world of commercial software. There are about 12 million software developers worldwide, and about $1,000 per coder is spent annually on productivity tools, Behlendorf reasons. He expects that $12 billion market to jump to $15 billion by 2004. So far, he adds, only about 1 percent of programmers practice open source-style coding.

“This industry is going to be huge. It already is huge,” he says. “That’s plenty of revenue for a number of open-source companies to go after.”

Commercial firms are also paying programmers to work on free and open-source projects at SourceForge. Jabber.com, for example, pays programmer Dave Smith to devote himself to the Gabber project. Jabber.com earns revenues by selling copies of Jabber and offering professional services to firms using the instant messaging program. Gabber could conceivably make Jabber.com’s business grow by increasing the number of potential Jabber users.

The full-time job at Jabber.com was a welcome surprise to Smith. Until he landed the gig last February, Smith worked as an information systems professional for a small company in Florida, and hacked away on open-source projects when he got home at night. He was amazed to find that the Denver company wanted him to work on Jabber-related software — at twice his salary.

“I never really thought I’d make a living doing open-source stuff,” he says.

Not everyone welcomes the kind of collaboration underway at SourceForge. Members of the Free Software Foundation, an older, radical wing of altruistic coders, argue that SourceForge effectively is stealing its thunder. Many software projects at the site are being built by coders sharing the foundation’s philosophy — namely, that all code should be freely published for the purposes of personal liberty and collective action. But SourceForge doesn’t properly promote this philosophy, says foundation leader Richard Stallman. Instead, the site is portrayed as part of the open-source movement, which focuses on revealing the source code behind software for the more practical purpose of creating better software than that built under a proprietary license.

“We don’t want people to assume we agree with the open-source movement simply because they see the words “open source” used to describe our project,” Stallman says. “Or (because) they see our project on a site that describes all it’s content as open source.”

Stallman says the Free Software Foundation approached SourceForge.net asking for both philosophies to be endorsed equally, but SourceForge officials didn’t step up to the plate.

“We’re committed to supporting both the free-software and open-source movements,” responds Patrick McGovern, SourceForge’s site director. He adds that SourceForge makes it easy to display the free-software license on project home pages and wants to work with the Free Software Foundation on ways to better promote its ideals. Stallman and his allies, though, are planning to build their own site. In an intriguing twist, they’re using the same software that powers SourceForge — it’s freely available to all, after all. The FSF hopes its site will recharge a philosophy that has been eclipsed in recent years with the advent of the open-source software movement and its more amiable attitude toward the traditional business world. Citing Raymond’s own research, Stallman says about a third of open-source programmers identify with free-software principles. And he hopes to attract those to the free-software collaboration site under construction.

Whether Stallman and the FSF will succeed in stealing back some thunder is open to question. There’s always the problem of how one pays for resources as such a site becomes more successful — a problem that will also continue to plague VA Linux as the company seeks profitability. Meanwhile, the number of open-source coders gathering at SourceForge continues to grow. And if the newcomers are anything like Julian Missig, the site is likely to keep taking the open-source movement further. Missig’s experience at SourceForge with the Gabber program has helped crystallize his college and career goals: He wants to study and eventually work on open-source user-interface programs.

The busy teen fully intends to keep cramming in his collective coding sessions.

“It’s something I look forward to,” he says. “It’s a lot of people collaborating on a piece of software to make it better.”

Free Photoshop for the people

Berkeley's Experimental Computing Club has produced some of the Net's most cherished software.

  • more
    • All Share Services

Free Photoshop for the people

Not long after the infamous Internet Worm nearly crippled the Net in November 1988, a University of California at Berkeley student was called to a U.S. government hearing in Maryland.

Phil Lapsley, co-founder of a student club at Berkeley called the eXperimental Computer Facility, had played an important part in the drama by helping to diagnose the worm and come up with a cure. The worm had taken advantage of a weakness in a popular version of the Unix operating system produced at Berkeley. Now officials from the National Computer Security Center and other government agencies were asking him about the episode — and getting an earful.

The young hacker blasted the federally funded Lawrence Livermore Lab for taking itself offline during the outbreak — a move that didn’t stop the infection but did cut the lab off from remedies sent from elsewhere on the Net. His criticism wasn’t entirely welcome, says Lapsley.

“This one woman from the Department of Energy said, ‘Forgive me, but we’re supposed to believe you? You’re some undergraduate from Berkeley.’”

“I said to her, ‘Whose computer operating system do you run?’” recalls Lapsley, “and she said, “Well, Berkeley’s.’ She sat down and I sat down.”

The episode neatly captures the spirit of the XCF, an organization that has directly or indirectly produced some of the most powerful and innovative open-source software of the past 15 years. The confident — some might say cocky — XCF undergraduates helped slay the Internet Worm, produced one of the first-ever Web browsers and developed two programs essential to the ecology of free software — the GTK tool kit (a set of tools useful for creating graphical user interfaces) and the GIMP, a Photoshop clone. Members of the XCF have also contributed code to the Gnutella file-trading project, a software program that many observers believe will be the successor to Napster.

“It’s almost like it’s our duty to create cool things for the world,” says Spencer Kimball, who co-wrote both the GIMP and the Unix versions of Gnutella.

All these achievements fit within the broader Berkeley record of producing free software critical to the rise and expansion of the Internet. Students in the XCF have added to the legacy created by well-known pioneers like Bill Joy, Kirk McKusick, Eric Allman and Sam Leffler. The success of the XCF’s lesser known hackers also offers some lessons worth considering. In contrast to the common perception that the act of programming is a solitary endeavor performed by lone cyber-cowboys, the XCF worked best when hackers were constantly poking their noses into each other’s code. And not all that politely, either — the XCF has a proud tradition of brutally honest peer review.

However, there is some question as to the future of the XCF. All members but one will be graduating this year, and it’s unclear whether future generations of Berkeley hackers will choose to gather in the hallowed XCF office. But the XCF has only its own success to blame. By helping to create software that made the virtual world rich and robust, it may have paved the way for its own real-world demise. The Internet now facilitates a vastly larger community of cooperating programmers than any single club can provide. And the open-source movement that so many XCF programmers have played a role in is now untethered to any geographic or physical limitation.

The XCF office is located on the ground floor of Soda Hall, a jade green building housing UC-Berkeley’s computer science department. It’s roughly 20 by 30 feet, and computer guts spill out onto a series of cluttered tables that are also dotted with intact machines and monitors. Posters cover the walls, including one that reads: “Need Unix Help? Have Programming Questions? The Doctors Are In.” A futon with a ratty orange blanket is crammed into one corner, next to a bathroom.

The office, used now by about eight current XCF members, gives a visitor little room to move about. But that very lack of space is one clue to the origin and culture of the club. UC-Berkeley has long suffered a shortage of rooms. When the university reorganized its turf in the mid-1980s, the existing computer center available to undergraduate hackers was set to be shut down. So Phil Lapsley and nine other computer enthusiasts wrote a proposal to establish an undergraduate-run facility that would both offer computer help to campus members and produce useful software projects.

The university agreed to the proposal, and in 1986 the XCF was born. Lapsley was the first director. He says his motivation for starting the club came from picking up coding tricks from other hackers at the earlier computer center.

“It just dawned on me, ‘My God, if you can just get all these people together in the same place, you can drastically increase how quickly people can learn,’” he says.

University administrators gave the XCF a half-dozen Sun Microsystems workstations — a coup at the time. The scarcity of powerful machines all but forced early XCFers to work together. Lapsley and Kurt Pires, another co-founder, came up with a course to teach fellow undergraduates the C computer language, which wasn’t offered by the university. The XCFers also helped one another figure out how to make their programs tighter, more elegant and more efficient.

The XCFers weren’t just tinkering around for fun. They were also under the gun to come up with programs that improved computing at the university, if not the entire world. That translated into a club admissions policy requiring would-be XCFers to propose a significant project. The pressure to keep the office also contributed to the culture of frank feedback. Often, XCFers spent more time on club projects than class work, and those who slacked off on their XCF work were asked to move on.

About one member a year would drop out, Lapsley recalls.

“I was one of the ones who got slapped around a lot,” says Jim Griffith, an early XCFer. “But it’s really hard to complain about being slapped down like that when it’s done in the pursuit of making you a better engineer.”

Griffith, 34, now works as a software engineer with Go.com, after spending five years with a map-related programming firm. That work drew on his XCF project, a graphical representation of U.S. Census data. He credits the XCF for significantly improving his skills.

“By the time I left, I knew more about Unix than just about anybody at Berkeley who wasn’t in the XCF,” he says.

It might not be a big stretch to say Griffith and other XCFers knew more about Unix than most people in the world in the late 1980s. Berkeley was ground zero for Unix expertise, having gained that distinction by spearheading the development of the BSD Unix operating system for the Defense Department. The Pentagon wanted a commonly accessible, free operating system for the research organizations linked together on Arpanet — the Internet’s predecessor.

Berkeley professor Bob Fabry had secured the contract for the work in the mid-1970s. He set up the Computer Science Research Group, which was spearheaded by Bill Joy and later by another graduate student named Kirk McKusick.

One of the hallmarks of the CSRG was welcoming coding improvements to Unix from a wide community of programmers throughout the world. McKusick says coders working together proved vital to the work.

“That was one of CSRG’s lasting legacies,” he says. “We set up a model that showed you could get 300 to 400 people to work together.”

Lapsley was one of those people when he first got to Berkeley — and the Internet Worm caper offered a perfect example of how cooperation could work. Lapsley and Pires were used to the occasional undergraduate trying to hack into their machines, and had set up the equivalent of trip-wire programs that would alert them to suspicious activity. So they noticed the worm hitting their machines at the beginning of the attack, the evening of Nov. 2. Peter Yee, another XCFer, quickly sent out an e-mail to the hardcore techies then on the pre-World Wide Web version of the Internet. Part of Yee’s e-mail was later quoted in Life magazine’s year-end issue: “We are under attack.”

But the XCF and the CSRG quickly went to battle stations. Members of both groups worked in concert to dissect the virus’ code, analyze it and write patches to neutralize it. Over the course of a few days, those vaccines were zapped out to panicking system administrators nationwide.

The Army, the Navy and the state of Florida all asked the XCFers for help. After staying up all night the evening of the attack, Lapsley walked back to his apartment dazed but proud.

“I remember being aware how this really momentous thing had happened, and the XCF was at the heart of it,” he says.

The worm victory wasn’t the only significant contribution by early XCFers. A club member named Pei Wei created Viola, one of the very first Web browsing programs. Lapsley also developed NNTP, a technology that still helps Internet users access Usenet newsgroups, and the hackers came up with some computer game advances, including an early multiuser Internet game called “xtrek” and an air-traffic control simulation.

After the early years of the XCF, both the club’s membership and production level ebbed and flowed. But another fertile period emerged in the mid- to late 1990s, when the XCF, led by Spencer Kimball and Peter Mattis, coauthors of the GIMP, proved that open-source software could compete with top-of-the-line proprietary applications.

The impetus for creating the Unix-based image-manipulation program was partly necessity, Mattis recalls. “I wanted to make a Web page at the time, and I couldn’t,” says Mattis. “It was that simple.”

But the project was anything but easy. It took the pair about two years. In the middle of the effort, Mattis decided he needed a better set of user interface software tools. So he wrote the GTK program, which in turn became a vital piece of code for building GNOME, one of the leading contenders for the role of a user-friendly desktop environment for Linux-based operating systems. Both the GIMP and GTK are now included in the standard versions of the Linux-based operating system distributed by Red Hat, TurboLinux and other major Linux providers.

By the late ’90s, university pressure on the XCF to produce or perish lessened. But the tough-love atmosphere continued. “Even if you did the greatest work you’ve ever done, people still would point out why it sucked,” says Kimball, who co-founded a Web portal-building firm called Wego with another XCFer.

XCFer Alice Zheng says the critical culture didn’t repel potential members. Instead, the harsh feedback helped feed the club’s coding quality, just as it had in earlier years. “Having an environment where you get direct feedback is important, because you don’t get that anywhere else” says Zheng, who is now getting her Ph.D. in artificial intelligence at Berkeley.

Zheng, Kimball, Mattis and others would spend as much as 80 percent of their time in the XCF office. And they came to see themselves as an elite breed apart.

“It had the highest concentration of motivated and capable individuals that I’ve ever seen,” Zheng says.

Excessive self-esteem? Perhaps. But XCFers from that era have an impressive track record. Of even greater significance, though, may be the role Gene Kan and Kimball have played in the success of file sharing software Gnutella. Gnutella, briefly released by AOL subsidiary NullSoft this spring and then yanked by corporate higher-ups, has been reverse-engineered by open-source programmers and disseminated throughout the Internet. As a potential example of the future of the Net — distributed file-sharing that depends on no central server for its operation — Gnutella could be hugely influential.

Kimball and Kan wrote a version of Gnutella for Unix. They also maintain a major Gnutella Web site and Kan has served as the primary spokesman for the software. From this bully pulpit, Kan has called Gnutella a defense against excessive government or corporate censorship.

“We’re headed toward a world where corporations control our lives — control the flow of ideas and the freedom to think,” he says.

Kan’s statement provides a link between two strands of Berkeley history over the past four decades: free-speech political activism and computer programming advances. Of course, the XCF isn’t as famous as Mario Savio’s Free Speechers or Bill Joy’s BSD Unix programmers. But the plucky student-run group does draw accolades from those who know its work.

Chris DiBona, the “Linux community evangelist” for VA Linux Systems and head of the Silicon Valley Linux Users’ Group, applauds both the GIMP and GTK. “Those are phenomenal projects with really great thinking behind them,” he says. DiBono estimates that hundreds of thousands, if not millions, of copies of the programs are in circulation.

XCF members “are a legend,” says Christos Papadimitriou, who was acting chairman of Berkeley’s electrical engineering and computer sciences department earlier this year. “They are forward-looking, activist and fiercely independent.”

On the other hand, the XCF’s high opinion of itself can irk some students in the computer science department.

“There’s the XCF’s pride,” says Daniel Silverstein, of the broader Computer Science Undergraduate Association. “There’s also, ‘What’s the XCF been doing since the GIMP?’”

Not much, admits current XCFer Eric Wagner. In recent years, the club has focused on more personal projects, often having to do with hardware. Members built a special MP3 player used during a Las Vegas road trip and also hacked into the XCF office climate-control system.

Wagner will be the only XCFer next year and is busy recruiting. That’s hard to do, though. Wagner says one problem is the ease with which student coders can do all their programming work online from home. And those hardcore Berkeley computer heads wanting to share programming tricks can now turn to the Web for a ready-made community. Sourceforge.net, for example, is home to more than 81,000 programmers working on 11,000 projects. Those who turn to Sourceforge can join or launch software projects small and large.

This type of online collaboration, of course, owes indirectly to XCF accomplishments: building trust in the Net by killing the Internet worm and reinforcing enthusiasm for the free-software movement with GTK, the GIMP and Gnutella. Yet it’s quite possible the XCF’s success may lead to its extinction. Still, Wagner is determined to preserve the club.

“This has such a history,” says the 20-year-old. “It’d be a shame if it dies just because I didn’t work hard enough.”

Whatever happens next in the annals of the XCF, Lapsley looks back on his now-grown baby with a father’s satisfaction. Currently the vice president of engineering of a Berkeley biometrics technology firm, the 34-year-old says he’s glad the XCF has produced valuable open-source software and improved people’s lives along the way.

He got a particular kick from discovering that GIMP files end in the letters “.xcf.” “It made me so proud,” he says. “That’s really cool.”

Continue Reading Close

The scourge of Silicon Valley

Anti-immigration crusader Norman Matloff says he's fighting for the rights of tech workers everywhere.

  • more
    • All Share Services

The scourge of Silicon Valley

You might call Norm Matloff a high-tech Don Quixote.

For the past seven years, the University of California at Davis computer science professor has been tilting his lance against Silicon Valley heavyweights and their hunger for more foreign guest workers. Foreign national techies working in the United States on “H-1B” visas not only depress the wages of U.S.-citizen programmers and squeeze out older engineers, argues Matloff, but also are often exploited along the way. Matloff has been tireless in his crusade. He has testified before Congress, written Op-Ed pieces, spoken with numerous reporters and zapped out countless e-mails railing against what he calls industry greed and shortsightedness.

But he’s losing the battle.

A bill to nearly double the number of skilled guest workers allowed annually sailed through Congress a few weeks ago. Despite the efforts of Matloff and a handful of other critics, the Senate agreed to raise the limit on H-1B visas to 195,000 by a 96-1 vote. The House passed the measure on a voice vote the same day, and on Tuesday President Clinton signed the bill into law.

It’s not the first time Matloff has had to lick his wounds. Shrugging off his testimony that tech firms are picky rather than parched for programmers, Congress also expanded the H-1B program in 1998. And to add insult to injury, the president of the Information Technology Association of America calls Matloff “president of the Flat-Earth Society.”

But there’s a case to be made that Matloff’s rants are on target and perhaps ahead of the curve. A heap of evidence — including a recent report by Congress’ own watchdog — casts doubt on the tech labor shortage and suggests both domestic programmers and H-1B workers have been hurt through the program. What’s more, Matloff’s claim that companies are “shooting themselves in the foot” with their hiring practices is taken seriously by at least some Silicon Valley firms and one top business school professor.

Matloff’s quest gets to the heart of some of the biggest questions facing the technology world: Is importing programmers and engineers good for the country’s long-term competitiveness, or does the practice dissuade Americans from pursuing technology and perhaps even push high-tech brain work overseas more than ever? And who should have our greatest sympathies? Domestic techies, who’ve enjoyed a better standard of living than most Americans, or coders from poorer nations, who may never get to taste the good life or stretch their minds fully if they can’t come to Silicon Valley?

Technology firms take the H-1B issue very seriously. Since 1999, the industry has coughed up $22 million to politicians, an amount double that given during 1997 and 1998. So Matloff doesn’t expect to win over Congress or the entire computer industry overnight. But he’s hopeful his logic eventually will prevail. And even though his long-fought battle against the likes of Sun, Oracle and Intel has earned him the label “the scourge of Silicon Valley,” Matloff would like nothing more than to reach a truce with the firms on the labor front and see them prosper.

“I’m their booster,” he insists. “I’m by no means a Luddite.”

Tech industry leaders don’t call Matloff a Luddite, but neither do they consider him to be helping their cause — or correct in his reasoning.

Matloff’s proclamations about abuse in the guest worker program only “perpetuate the myths surrounding the H-1B visa program,” wrote George Scalise, president of the Semiconductor Industry Association, in a September letter to the editor of the Washington Post.

Harris Miller, president of the Information Technology Association of America, says Matloff’s 1998 predictions of gloom and doom for domestic workers haven’t panned out. Just look at unemployment, says Miller. In 1999, 2.3 percent of programmers were unemployed, well below the nationwide unemployment rate of 4.2 percent — itself an extremely small figure historically.

“He’s obviously a very good computer science professor,” Miller says. “But on this issue he’s just flat wrong.”

Matloff has evidence suggesting that unemployment figures don’t capture older techies who’ve been forced out of the field. But being asked to address Miller’s charge elicits more than just the facts. During an interview at a Chinese restaurant in Oakland, Calif., Matloff’s voice rises when he considers Miller and other industry spokesmen.

“Lobbyists are paid to be sharks,” he says.

It’s a rare angry moment during the dim sum lunch. For the most part, Matloff is soft-spoken as he speaks with me and orders dishes in fluent Chinese. His mild-mannered temperament, along with his small, wiry frame and thick glasses, fits the stereotype of a computer nerd perfectly.

Only there’s a big difference between him and most coders, he says.

“Programmers tend to be reserved people,” observes Matloff, “not types who would write to their representatives, let alone carry a picket sign.”

Matloff’s activism on the H-1B issue is part of a broader social justice streak. He’s long been interested in ethnic minority issues, and served a stint as head of UC-Davis’ affirmative action program. Matloff married a Chinese woman who eventually became a U.S. citizen, and he used his e-mail network to spread awareness of the Wen Ho Lee case.

Matloff’s concern for the underdog comes packaged with a hungry, disciplined mind. He taught himself both Cantonese and Mandarin as an adult by listening to the radio and bugging friends for help. He also taught himself how to program computers. His Ph.D. from UCLA is in theoretical math.

It’s not surprising, then, that Matloff has developed an impressive lay expertise on the subjects of age discrimination and the H-1B visa. One sign of the volume of his knowledge: The testimony he gave to Congress and that he continually updates has grown to 110 pages.

Labor unions, engineer associations and other critics have opposed the H-1B program. But Matloff has been central, says Mark Krikorian, executive director of the Center for Immigration Studies, a nonpartisan Washington think tank.

And although the H-1B expansion passed overwhelmingly, Matloff’s efforts kept the measure from being worse, Krikorian says. In particular, Matloff’s credibility and activism helped kill an exemption for foreign graduate students at U.S. universities — a feature that could have allowed an additional 100,000 or more H-1B workers annually, Krikorian says.

“He can’t be pigeonholed as a gadfly,” Krikorian says. “He knows more about this than just about anybody. He’s intimately familiar with the personal, professional and academic aspects of the issue. And you can’t write him off as a nut.”

Matloff’s ambitious quest started simply enough: as a professor concerned about his students. In 1993, Silicon Valley hadn’t yet climbed out of a recession, and computer science grads from UC-Davis experienced the tough times firsthand.

“We had graduates working in downtown San Francisco, at Macy’s as salesclerks,” Matloff recalls.

The fact that Congress was making it harder on his students by importing tech talent from abroad was especially maddening to Matloff. Washington had created the H-1B program in 1990.

The program works like this: Skilled foreign workers — typically those with a bachelor’s degree or higher — are allowed into the United States for a period of up to six years. The latest statistics from the Immigration and Naturalization Service show that 47 percent of those who enter are systems analysts or programmers and an additional 5 percent are electrical or electronics engineers.

The program started off with an annual cap of 65,000. But with the high-tech industry lobbying for more workers, the limit was raised in 1998 to 115,000 for this year. The recent legislation will raise the annual ceiling to 195,000 until 2003. Other features of the new law include giving H-1B workers greater job mobility and using a $500 application fee per guest worker to provide more than 60,000 college scholarships for U.S. students in tech-related fields. Industry leaders point out that H-1B visas have run out each year since 1997. The most recent study on the issue, released by the Information Technology Association of America, concluded that 1.6 million new information technology jobs would be created this year, and half would go unfilled.

But the tech industry hasn’t relied just on its reports to win over Congress. According to the Center for Responsive Politics, the computer equipment and services industry has contributed $12 million to Democrats and $10 million to Republicans since 1999. The total more than doubles the amount donated by the industry in 1997 and 1998.

“Congress has been bought off equally,” Matloff says. “Neither party dares to cross the industry.”

That may sound like extreme rhetoric, but the evidence on his side does make the 96-1 Senate vote rather puzzling.

One of the most impressive documents in Matloff’s arsenal is a September report by the U.S. General Accounting Office. Titled “H-1B Foreign Workers: Better Controls Needed to Help Employers and Protect Workers,” the report concluded that existing labor market studies “do not permit a conclusion as to the extent of any IT skill shortage.” The report said the labor market studies had flaws including a lack of data on whether companies considered jobs filled by contractors as vacancies.

Then there’s Matloff own research challenging the notion of a “shortage” of IT workers. By studying a database of college graduate surveys, he found that only 19 percent of computer science grads are still in that field 20 years later — compared with 52 percent of civil engineers. To him, the low unemployment rate for programmers masks what he sees as a pattern of older techies getting left out in the cold in the middle of a red-hot economy.

Research by the Institute of Electrical and Electronic Engineers supports Matloff’s claim that veteran coders are being driven out of the field. In a 1998 report, the institute discovered that unemployed members typically require three additional weeks to find a new job for each year of age over 45.

“Many employers find H-1B [pre-green card] programmers and engineers attractive because they will accept lower salaries and poor working conditions,” Matloff wrote in a paper titled “Debunking the Myth of a Desperate Software Labor Shortage.”

This document might be called the Matloff bible. Originally his testimony to Congress in 1998, he last updated it on Saturday. One section of the paper describes Matloff’s research into companies’ alleged severe need for workers. He checked with a total of 25 high-tech firms and found that they typically offer jobs to just 2 percent of applicants and a quarter of interviewees.

Matloff also notes out that tech wages haven’t skyrocketed as one might expect in a labor shortage. One of his recent e-mails cites an Oct. 2 Newsweek chart showing that the starting salaries for computer science graduates rose only 20 percent from 1995 to 1999, a smaller increase than in the fields of business administration, accounting and sales/marketing.

Despite the current tech boom, Matloff’s own students are having trouble finding good work. Fewer than half of UC-Davis computer science graduates get programming jobs.

“They put in all this work,” says Matloff, “then they’re given a job in customer service,” he says. “They’re not using what they’ve learned.”

Matloff also has become a one-man information clearinghouse for veteran programmers angry with the H-1B program. The vast majority of the people on Matloff’s e-mail list are techies, who each day feed Matloff 20 to 30 e-mails with articles on the H-1B program and tales from the trenches. Matloff can give reporters the names of dozens of programmers who’ve gone months or years without work.

One techie connected to Matloff is 45-year-old programmer Rob Sanchez. A Phoenix resident with eight and a half years in military electronics, Sanchez lost his job about a year and a half ago, on the very day his company brought on an H-1B worker to do software programming.

Sanchez then spent 18 months looking in vain for tech work. He finally landed a job as a programmer for a communications company two months ago. But he blames age discrimination, exacerbated by the H-1B program, for his difficulties.

“Your professional career is being undermined by our own government and rich corporate lobbyists,” he proclaims on his Web site.

Matloff often makes reference to Sanchez’s Web site, because it includes a database of H-1B salaries Sanchez obtained through the Freedom of Information Act.

At Sanchez’s site you can learn, for example, that Microsoft planned to pay an H-1B software engineer $34,000 in 1997, when the average computer engineer salary for that year was $56,590, according to the Labor Department.

A Microsoft spokesman said the salary of a particular guest worker might reflect the average of his colleagues at Microsoft rather than the industrywide average.

Microsoft’s position is perfectly legal. But it highlights the point made by Matloff and others that guest workers frequently are mistreated by the H-1B program.

On this issue, once again, the GAO sides with Matloff. Its September report documents numerous cases of fraudulent applications and INS policy that discourage critical review of applications. What’s more, the program is ripe for abuse by employers. The GAO noted that while companies have to pay guest workers a “prevailing wage,” firms can use almost any source to determine that wage and the Labor Department is compelled to accept it.

The GAO also reported that the H-1B program creates a relationship between guest worker and employer that discourages whistle-blowing. H-1B workers are dependent on employers for their stay in this country, and many come with the expectation that their company will file a green card application on their behalf.

Matloff argues that the H-1B system has amounted to a form of “indentured servitude.” Besides the GAO report, press accounts back up this assessment. In the past few years, newspapers including the Baltimore Sun and San Francisco Chronicle have written about the poor treatment received by H-1B workers. The worst abuses seem to occur at so-called body shops — consulting outfits that outsource programmers to other firms. Body shops have been described as “benching” the guest workers, an illegal practice where they are not paid their full salary unless the firm has contract work.

The new legislation addresses some of the program’s flaws. For example, H-1B workers will be able to switch employers more quickly. And those with long delays in the processing of their green card applications will be able to transfer to a new company.

But Matloff sees the potential for continued exploitation. It will still be possible for employers sponsoring a guest worker for a green card to foot-drag in the initial stages of the process. And even without foot-dragging, foreign workers are still effectively beholden to a firm for at least three to four years, he estimates.

Matloff admits the H-1B workers may earn much more here than they ever would in their homelands. But he draws an analogy to immigrant workers toiling in U.S. garment shops. “They’re being exploited by American standards,” he says.

Many H-1B workers beg to differ, particularly those who come from from less-developed nations. Indians are by far the single largest group of guest workers — they make up nearly 43 percent of total H-1B applications — and their stories often are dramatic. A computer programmer in India, for example, might make the equivalent of $10,000 to $15,000 a year. H-1B workers in computer-related jobs, by contrast, were slated to be paid a median wage of $53,000. And the salaries for some H-1B techies can soar to $80,000 or more.

But money isn’t the only magnet for foreign engineers. The chance to do computer work on the cutting edge is also a huge draw, says Jayaram Manda, a 28-year-old H-1B programmer from India. Despite a burgeoning technology industry in Indian cities like Bangalore, the United States is the place graduates from India’s prestigious universities want to go.

“This is the technology world,” says Manda, who lives in the San Francisco Bay Area and works for a firm called the Consilium Group. “You read about something in the textbook, and you want to implement it.”

While they may enjoy their work here, H-1B workers can face resentment from domestic techies. Aditya Agrawal, an H-1B chip designer at a Silicon Valley firm, has had colleagues accuse him of cutting into domestic wages and contributing to age discrimination.

“This is a tough industry. Tough things get said all over the place,” he says.

Agrawal, 31, has an answer for his critics. He believes the H-1B program has kept U.S. tech salaries from falling, because the foreign workers have fueled Silicon Valley’s success over the past decade. Without these skilled workers moving here, the software or Internet industries in places like Taiwan or India could have blossomed, he argues.

“I doubt it could have happened [here] without the best of the best coming here and working together to get things done,” he says.

Matloff doesn’t share Agrawal’s theory. First, he disputes the idea that the “best and the brightest” arrive via the H-1B program. Citing an Immigration and Naturalization Service audit, he notes that many H-1B applications are fraudulent. Matloff also argues that the vast majority of H-1B workers have salaries well below those typically commanded by geniuses — at or above $100,000 — and points out that only nine of 115 computer-related awards given by the Institute of Electrical and Electronic Engineers were given to immigrants.

Second, U.S. companies would move more software development work offshore if they could. But Silicon Valley succeeds partly because of the face-to-face contact among colleagues, Matloff argues.

Matloff and the tech industry have radically divergent views on how the H-1B program fits into global capitalism. To the industry, H-1B workers are critical to maintaining America’s technological edge, and therefore to preserving high-paying jobs here.

But Matloff suggests the country is undermining its tech industry with guest workers. He points to the work of Vernon Briggs, a Cornell labor economist who believes the expanded H-1B program will dissuade American kids from coding careers — because all the extra techies means wages won’t rise.

And in a recent e-mail to his electronic flock, Matloff suggested that H-1B workers may play a more immediate role in the country’s losing technology work. He distributed an article describing how H-1B workers tied to a worldwide consulting firm can provide a foothold into the United States for work done overseas. Another story Matloff sent around quoted an Indian software industry official as expecting the expanded H-1B program to help boost Indian software exports.

That’s “another perverse effect of the H-1B program,” Matloff wrote.

Matloff doesn’t just blast the H-1B program and companies using it. He has a solution in mind, and Silicon Valley might do well to listen. As Matloff sees it, not all companies are using guest workers out of greed. Some simply believe they can’t find qualified workers. And this is the rub: He thinks firms have developed a shortsighted attitude that programmers and other techies must come with experience in the latest, highly specific skills. That’s why firms reject the vast majority of their candidates.

Matloff suggests hiring based on overall coding prowess instead, and letting techies learn on the job.

“Any competent programmer experienced in the C language (the standard for the past 15 years) can become productive in Java in a couple of weeks,” he wrote in a recent essay for Forbes.

Tech-firm choosiness is costly — Matloff writes that the average time it takes to fill a job in Silicon Valley is 3.7 months. That’s forever in today’s fast-paced economy. Matloff adds that the obsession with skills leads to pumped-up salaries for specialists and promotes job hopping, which can cause chaos for companies.

You’d think America’s richest, most powerful firms wouldn’t need the business advice of a computer scientist. But Matloff’s analysis is shared in large part by Peter Cappelli, a professor at the University of Pennsylvania’s Wharton School.

Cappelli published a paper on the IT labor issue this summer, and agreed that hiring on the basis of a programmer’s coding quality is key. But firms have been poor judges of programming talent, and then depend too heavily on résumés,” Cappelli observes.

“In the absence of a good understanding as to what predicts jobs performance, employers rely on credentials, especially academic credentials, which are highly visible and easily contribute to bidding wars among competitors,” Cappelli writes.

Cappelli sympathizes with companies that have trouble finding the IT talent they need, especially as technologies change rapidly. But rather than import techies, Cappelli concludes, the U.S. should concentrate on keeping so many domestic ones from leaving the field. He calls for better management for often-isolated coders, higher pay for top guns — who are more than 10 times more productive than poor programmers — and more retraining programs for veteran employees.

Matloff’s ideas also are echoing in Silicon Valley. A few companies have even approached him asking for hiring advice. Matloff helped persuade one semiconductor firm to change its policy of considering only graduates with straight-A academic records.

With all his advocacy work, Matloff has become a hero of sorts for domestic techies. One of his fans is Terry Oldberg, a 60-year-old engineer who spent eight of the past 13 years unemployed.

“He was essentially a one-man opposition to the huge juggernaut of the business lobby,” says Oldberg, also the chair of the Northern California chapter of the Programmers Guild. “He is truthful and very courageous.”

Matloff now feels a responsibility to people like Oldberg, but he doesn’t feel like a hero. He has never faced any heat from his university department for his activism. And although Matloff sends out as many as six e-mails a day commenting on news articles, he claims he gets this work done in a less than an hour a day.

He’d like to do more research on where older techies go after leaving the field and what happens to their pay. But he’s careful not to burn out or lose sight of other priorities. He’s determined to keep up his full-time professor duties, raise his 8-year-old daughter in Walnut Creek, Calif., and spend time with his wife, who just retired from a programming career.

And unlike some activists who develop a burning hate for their opponent, Matloff remains a geek at heart. He’s out to reform tech firms, not destroy them. And he believes they’ll ultimately see that the narrow way they hire and fire does not compute.

“I still believe that eventually they’re going to see it’s in their own best interests to broaden the way they hire,” he says.

What’s more, despite the lopsided H-1B vote, Matloff hasn’t given up on Washington. He believes enough programmers may get hurt by the H-1B program that members of Congress themselves will feel the pinch — perhaps the computer engineer daughter of a senator won’t be able to find a job.

So he plans to keep consulting tech firms, talking to the press, updating his online paper and sending out e-mails. Rather than a hopeless dreamer, he sees himself as Jimmy Stewart-like, repeating his point against the odds until people believe.

“It’s not ‘Don Quixote,’” he insists. “It’s ‘Mr. Smith Goes to Washington.’”

Continue Reading Close