Why software still stinks

Programming must change -- but how? At a reunion of coding pioneers, answers abound.

Published March 19, 2004 8:30PM (EST)

In some quarters today, it's still a controversial proposition to argue that computer programming is an art as well as a science. But 20 years ago, when Microsoft Press editor Susan Lammers assembled a collection of interviews with software pioneers into a book titled "Programmers at Work," the idea was downright outlandish. Programming had long been viewed as the domain of corporate engineers and university computer scientists. But in the first flush of the personal computer era, the role of software innovator began to evolve into something more like the grand American tradition of the basement inventor -- with a dollop of the huckster on top and, underneath, a deep foundation of idealism.

It made sense that the people writing the most important code for the new desktop machines were ragged individualists with eccentric streaks. At a panel on Tuesday (sponsored by the SDWest conference and Dr. Dobb's Journal) that celebrated Lammers' book, seven of the 19 original subjects of "Programmers at Work" lined up on stage to talk about what's changed in software over the past two decades -- and demonstrate that they have lost none of their cantankerous edge.

In "Programmers at Work," Lammers told the crowd, "I looked at the programmer as an individual on a quest to create something new that would change the world." Certainly, the panel's group lived up to that billing: it included Andy Hertzfeld, who wrote much of the original Macintosh operating system and is now chronicling that saga at Folklore.org; Jef Raskin, who created the original concept for the Macintosh; Charles Simonyi, a Xerox PARC veteran and two-decade Microsoft code guru responsible for much of today's Office suite; Dan Bricklin, co-creator of VisiCalc, the pioneering spreadsheet program; virtual-reality pioneer Jaron Lanier; gaming pioneer Scott Kim; and Robert Carr, father of Ashton-Tate's Framework.

But for all their considerable achievements, this was not a group content to snooze on a heap of laurels. In fact, though the hour-and-a-half discussion was full of contention, one thing all the participants agreed on was that software today is in dire need of help. It's still too hard: not only for users struggling to make sense of poorly designed interfaces, but for programmers swimming upstream against a current of constraints that numb creativity and drown innovation.

These veterans shared a starting-point assumption that the rest of the world is only slowly beginning to understand: While computer hardware seems to advance according to the exponential upward curve known as Moore's Law (doubling in speed -- or halving in cost -- every year or two), software, when it advances at all, seems to move at a more leisurely linear pace.

As Lanier said, "Software inefficiency can always outpace Moore's Law. Moore's Law isn't a match for our bad coding."

The impact of this differential is not simply a matter of which industry gets to collect more profits. It sets a maddening limit on how much good we can expect information technology to achieve. If computers are, as it has often been put, "amplifiers for our brains," then software's limitations cap the volume way too low. Or, in Simonyi's words, "Software as we know it is the bottleneck on the digital horn of plenty."

Most successful programmers are at heart can-do engineers who are optimistic that every problem has a solution. So it was only natural that, even in this relatively small gathering of software pioneers, there were multiple, and conflicting, ideas about how we should proceed in order to break that bottleneck.

Simonyi believes the answer is to unshackle the design of software from the details of implementation in code. "There are two meanings to software design," he explained on Tuesday. "One is, designing the artifact we're trying to implement. The other is the sheer software engineering to make that artifact come into being. I believe these are two separate roles -- the subject matter expert and the software engineer."

Giving the former group tools to shape software will transform the landscape, according to Simonyi. Otherwise, you're stuck in the unsatisfactory present, where the people who know the most about what the software is supposed to accomplish can't directly shape the software itself: All they can do is "make a humble request to the programmer." Simonyi left Microsoft in 2002 to start a new company, Intentional Software, aimed at turning this vision into something concrete.

Frederick Brooks, in a famous 1987 essay, declared that the prospect for a programming "silver bullet" -- to slay, once and for all, the monster-like characteristics of so many software development projects -- was dim. But Simonyi said he believes his project will provide that very silver bullet.

You can't fault him for ambition. Or can you? Jaron Lanier, sitting appropriately at the opposite end of the stage from Simonyi, argued that there's a deeper failure of vision in the software world that requires even more radical change. "A lot of stuff in the Mac and Windows world was supposed to be temporary and got wedged into place," he said. "Making programming fundamentally better might be the single most important challenge we face -- and the most difficult one." Today's software world is simply too "brittle" -- one tiny error and everything grinds to a halt: "We're constantly teetering on the edge of catastrophe." Nature and biological systems are much more flexible, adaptable and forgiving, and we should look to them for new answers. "The path forward is being biomimetic."

What about the open-source movement, which over the past decade has won considerable loyalty and enthusiasm in many programming quarters?

"There's this wonderful outpouring of creativity in the open-source world," Lanier said. "So what do they make -- another version of Unix?"

Jef Raskin jumped in. "And what do they put on top of it? Another Windows!"

"What are they thinking?" Lanier continued. "Why is the idealism just about how the code is shared -- what about idealism about the code itself?"

At this point, Andy Hertzfeld, who has devoted himself in recent years to open-source projects like Eazel and Chandler, spoke up for the maligned legions of Linux-heads. "It's because they want people to use the stuff!"

His comment underscored something that's frequently misunderstood about the open-source approach, which is often wrongly stereotyped as loopily communal and out-of-touch with business reality. There's an essential pragmatism to the notion that programmers work best when they can share, and learn from, one another's work. After all, every other field of human endeavor works that way.

Bricklin sent waves of laughter through the auditorium by reading a passage from Lammers' interview with Bill Gates in which the young Microsoft founder explained that his work on different versions of Microsoft's BASIC compiler was shaped by looking at how other programmers had gone about the same task. Gates went on to say that young programmers don't need computer science degrees: "The best way to prepare is to write programs, and to study great programs that other people have written. In my case, I went to the garbage cans at the Computer Science Center and I fished out listings of their operating systems."

Bricklin finished reading Gates' words and announced, with an impish smile, "This is where Gates and [Richard] Stallman agree!"

The "Programmers at Work" panelists were full of optimism about new opportunities to reinvent software -- in the mobile-phone world (where, Scott Kim noted, the constraints of small screens and tiny memory made it feel "like the early days" again), in the new universe of RF tags, and in the still-unfolding saga of global networking. Bob Carr reminded everyone that technology transformations usually take 20 years to unfold -- "I remember thinking in 1987 that the PC industry was mature, it was over" -- and that the Internet is only halfway through that cycle.

Still, that picture of Bill Gates dumpster-diving for operating-system code was hard to shake. Finding new ways to think about programming and to make better software demands a willingness for pioneers to open up their work so others can learn from it. "Getting the software industry on a more open, fair and level playing field," as Hertzfeld put it, is a prerequisite for any leap forward in the programming world. Software patents are a looming train wreck; competition in most "end-user" software is largely a distant memory. Simonyi's technical bottleneck is also a social, political and business logjam.

In the era of "Programmers at Work," it was possible to imagine the lone-hero programmer as a genius operating beyond the reach of political and social forces. Today, even the best programmers can't ignore the vast web of interdependence their own work has helped shape.

By Scott Rosenberg

Salon co-founder Scott Rosenberg is director of MediaBugs.org. He is the author of "Say Everything" and Dreaming in Code and blogs at Wordyard.com.

MORE FROM Scott Rosenberg

Related Topics ------------------------------------------