Software is hard

How do you define "new" in terms of a software project?

In every big software project today, there are always areas that are repeating stuff that other people have already done, and then there are the parts of it that are the new parts, that are the reason you are writing the software -- because if there weren't any new parts there would be no point in building the software. But it's a good question, because this is where the mind-set of developers affects the process and where projects often get into trouble. You're doing the project because there is this new feature or features that you need. The developers will often look at this and say, well, we know we have to do that, but then there is all this other stuff that needs to be done: For a content management system, you need an interface for putting in the text, you need a database, all these basic things. The developer might say, I could take this thing off the shelf that exists already and plug it in, but it's going to take almost as long for me to learn how to do that, or maybe even longer to learn it, than to write it myself.

And programmers, as I quote Larry Constantine in my book, programmers are programmers because they like to code -- given a choice between learning someone else's code and just sitting down and writing their own, they will always do the latter. And the programmer who says, it will be faster for me to write it, rather than to learn it, is usually correct. Except that what he will write, most likely, is something that will work but will not have its rough edges worked out, will not have the benefits of a piece of software that has actually been used for a few years, where the bugs have been found and the users have given feedback and have helped you figure out where the problems are. So what they will often be handing you at the end of that I-can-do-it-faster-myself thing is something that works, but that is kind of a mess in certain ways. Whereas the thing that you were going to pull off the shelf, maybe it will take the programmers a while to learn it, but once they learn it enough to hook it up to this project you are creating, what they are hooking up will probably have a lot fewer problems.

You always hear, with regard to software, that managing developers is like herding cats, and what you just said kind of alludes to that. Does that mean one of the problems with getting software projects done on time is the psychology of the people who are actually doing it?

That's a huge and fascinating subject. There is this raw fact in the field that the best way to create software would always be to have just one person write it. At Salon, our original content management system, the predecessor to the big one that was a disaster, was a much smaller system that one developer wrote. It was limited, but the process of creating it was much less painful.

So then we went from one developer to a team of a half dozen people. One person can keep it all in his or her head. But the moment you go to two people or beyond you introduce all of the possibilities for miscommunication between people. That is one of the things that readers of "Dreaming in Code" will get a sense of from the story. I have huge respect for the developers of Chandler, led by Mitch Kapor. They are all really good at what they do. There wasn't a lot of internal politicking, it was a pretty straightforward project in a lot of ways, and yet they had great difficulty getting on the same page, just on things like the terminology they were using.

There were some unique design concepts that they were trying to introduce to the world of personal information management, and just coming up with language to describe those things, in such a way that the software could be developed by multiple developers, turns out to be really hard. And that is less a function of the mentality of developers than it is of the insubstantiality of the product of software -- the fact that it is made up, it's layers of stuff that have been made up on top of other layers of stuff that have been made up, all the way down to the bottom.

Is there a sense of progress in the field? Do people know now how to do things better than they did in 1980?

I think there is a sense of progress, because clearly we have programs that do more extensive things today and that are used more widely by more people. And there are things that programmers never stop to think about today that they had to worry about 30 years ago. For better or worse, and there are some people who say it's worse, a beginning programmer today does not learn any machine code. That has been abstracted out of the picture, and a lot of even the lower-level stuff that is above machine code that programmers used to learn about they don't learn about today. There are some people who argue that that's a problem, but most of the time, it enables people to get more done faster.

But it is slow progress. It's linear progress.

You followed the Chandler project for four years. Was it frustrating to see the project get delayed beyond the time frame of your book deadline? Or was the fact that they didn't finish on schedule a fitting example of exactly what you were writing about?

I hope that readers won't feel dissatisfied or frustrated. I feel like it's still dramatic, but it's drama like "Waiting for Godot." I've talked to Mitch Kapor and the other people there, and I know they are a little frustrated with how that happened, because they feel like they've made a lot of progress since the end of my story. They are going to do what's called a "preview release" of Chandler in April. I'm excited about that -- I still want to use that program that they dreamed about building. But I felt that I had reached a point where, for the questions I had started with, I had as good answers as I was going to get. And the open-endedness of the book's story, I feel, is actually a reflection of the book's themes.

You recently spoke at Yahoo to more than a hundred people. Were they expecting you to help solve their software development problems?

I hope not! No, I don't think they were expecting that, and anyway, I certainly didn't solve their problems. When I started writing this book, I knew right off the bat that I needed to be really clear with people that I wasn't writing a how-to book, and I wasn't writing a book of advice. First of all, I don't think I'm really qualified, and second, the shelves are groaning with those books. That wasn't my goal.

My goal was really more to write something that, if you were a developer, then yes, you might find it interesting. But even more, if you had a relative who was always wondering, "What is it that you do all day?" you could hand my book to that relative and say, This is what my work is really like.

Story finder (3 ways to search Salon)

Search terms:

Salon the Web

Salon Directory (browse by topic)