Simson Garfinkel

One of Raskin’s early observations is that people quickly become habituated to routine processes and procedures. This works to both the advantage and the detriment of the interface designer. Habituation lets an experienced person use a well-designed interface more quickly. But that same habituation can also lead to errors — sometimes catastrophic ones. And that’s when it would be great if we had a truly workable undo.

Consider the Yes/No or OK/Cancel questions that many computer systems ask. “Do you really want to empty your trash? (Y/N)” “Are you sure you want to permanently delete the selected items?” People become so accustomed to these questions and pop-up boxes, Raskin writes, that after seeing them a few times they habitually click “OK,” even when they should click “Cancel.” Hence, they click right through something like: “WARNING: All data on non-removable Disk Drive C: will be lost. Proceed with format? (Y/N)”

Far better than giving the user a Yes/No or OK/Cancel choice would be to create a general undo facility that worked consistently throughout the entire computer system. When, weary from slaving for hours on that paper you’re writing, you mistakenly tell your word processor to shut down without saving the final version, you don’t want it to ask, “Are you sure?” You want it, when you realize your mistake, to promptly undelete any work you’ve foolishly trashed.

Raskin’s words became especially poignant to me last month, when a minor user-interface tick and an OK/Cancel alert caused me to lose the minutes of a board meeting that I had been taking on my Palm Pilot. It was an hour into the meeting, and one of the organization’s board members asked me to beam my minutes into her Palm VII. Trying to be helpful, I clicked the button to display my computer’s pull-down menu, selected “Beam Memo” and was prompted with a pop-up box asking: “Beam current memo? OK/Cancel.” I clicked “OK” and suddenly the memo vanished.

Of course, I hadn’t clicked “Beam Memo” on the Pilot’s menu, but “Delete Memo.” As Raskin notes, I had been so focused on the idea of beaming my memo to the board member that I had misread the confirmation box — a box that was designed to prevent me from doing precisely what I had then proceeded to do.

Instead of a confirmation box, a far better design would be to have the computer always delete the menu, but then to allow the ability to undo the last action. The ubiquitous OK/Cancel box is a terrible user-interface design, writes Raskin, because it slows you down the majority of the times you are actually trying to do something, and the few times that you really, really need the confirmation box — when you are habituated to a user interface and about to make a mistake that will cause you to irrevocably lose data — you don’t stop to read it. You don’t stop because you have become habituated.

Of course, the Palm operating system does have an undo feature. Unfortunately, it doesn’t work all the time. Undo works for undoing modification to text in the memo pad application, for example, but it can’t undo the deletion of a memo. There’s an undo option on the Palm’s appointment book program, but it can’t undo changes you might make to an appointment’s date or time. These limitations aren’t the result of the Palm’s low-powered microprocessor or small memory; they’re the result of poor design — poor design of both the memo pad application and the underlying operating system. And they are design problems shared by many systems.

Speaking as a programmer and as a designer, creating a generalized undo feature is hard work. To do it, you must remember every change that affects the user’s data so that you can undo those changes if the user asks. Few application frameworks provide an undo facility, so each programming team has to create its own. Although this shouldn’t be hard to do in principle, in practice it enforces a discipline that few of today’s programmers are up to. One of the reasons, I think, is that they lack good examples: Since no program currently on the market today does undo properly, there is little incentive for other programmers to do better.

Consider the undo feature in Microsoft Word. Overall it’s pretty good, but it frequently behaves in an unpredictable manner. For example: Type a paragraph of text. Select the paragraph with your mouse and choose the “Copy” command. Now select the last sentence of the paragraph and choose the “Cut” command. Now click undo, click the mouse at the end of the paragraph and choose the command “Paste.” What happens? You should get the entire paragraph, but, instead, you get just the last sentence. That’s because Microsoft’s undo doesn’t really undo your last command; instead, it reverts to the last change to your document. In this example, the last command also affected the clipboard, which Word’s undo command doesn’t restore.

Many applications don’t even have an undo facility. Last year, for instance, I received an e-mail from a reader who was furious at Intuit. The reader had lost a significant amount of time because Intuit’s Quicken lacks an undo feature and he had inadvertently made a change to a transaction in his checking register. Of course, he didn’t know what the change was — it was, after all, inadvertent — and he spent several hours trying to figure out why a reconciled transaction had disappeared but his register still balanced. The man eventually discovered that he had changed the year of a credit card charge from 1999 to 1909. An undo feature that would undo a change to the last transaction would have saved him much work and frustration.

The computer industry has technical standards that describe everything from the voltage transmitted on an Ethernet cable to procedures that companies must follow for ensuring the “quality” of their products. But few standards ensure that these products will be usable or, to use Raskin’s word of choice, humane. Building an undo feature that always works would be a good place to start.

The combination of the Visor PDA and phone does a better job than all of these other attempts — it’s even better than the ill-fated pdQ. And in a curious twist, the Visor duo makes a more usable cellphone than most stand-alone cellphones. It’s a real example of integration creating a better product — something that the high-tech industry constantly promises, but rarely delivers.

The original Handspring Visor was designed with the VisorPhone in mind. Back when the folks at Handspring (incidentally, the same people who created the original Palm organizer) were designing the first Visors, they wanted to create a phone module but simply didn’t have the time or the engineering resources. So they did the next best thing — they designed into the Visor the connectors and additional hardware that the future phone module would require. As a result, the first Visors to come off the line had a little hole in the lower left-hand corner, with a tiny microphone wired in place. But the microphone wasn’t wired to anything inside the Visor itself; instead, a pair of wires snakes from the microphone to the Visor’s “springboard” expansion slot. When you plug the VisorPhone module into the Visor, it uses the built-in mike — which sounds great.

Likewise, a second pair of wires goes from the connector at the base of the Visor to the springboard and is used for charging the phone’s batteries (which requires much more power than can be provided over a standard Universal Serial Bus interface). The speaker for the VisorPhone is in the black module itself. (There is also a jack for a headset.)

Because of this advance planning, the Visor and VisorPhone combo is significantly smaller than other ill-fated integration attempts. My pdQ was a big brick that was awkward to carry; worse, the pdQ had two different user interfaces — a keypad for the phone and the LCD screen for the Palm. The new combo, on the other hand, weighs 8.5 ounces and fits in my shirt pocket. Better still, the VisorPhone is completely controlled by a set of PalmOS applications that automatically get installed on the Visor when the module is inserted.

Making phone calls is pretty easy. You start by clicking the “phone” button on top of the phone module, which makes the Visor display an application with 10 auto-dial buttons. Touch a button with the Visor stylus (or your finger) and the phone dials. If the phone number you want to call isn’t in your auto-dialer, you click the “phone” button again and the Visor will display a telephone keypad; you tap the number and then talk. You can also dial any phone number in the Palm’s address book by tapping the number displayed and then confirming that you wish to dial it. Finally, you can dial any number in a memo or on your calendar by selecting the number with the stylus and then pressing the “dial” button.

The Visor displays the call’s elapsed time, and you can put a person on hold by tapping a button on the phone application. You can also place a second call or handle a second incoming call. Or, if you are exceptionally good at multitasking, you can click into another PalmOS application. You can even turn off the Palm, since the phone has its own battery and built-in microcomputer. Just about the only thing you can’t do is pull the VisorPhone out of the Visor’s springboard slot — that makes it hang up.

But voice calls are just the beginning of the action. The second button at the top of the module is for sending and receiving SMS messages. (SMS stands for “short messaging service,” and is built into the GSM the VisorPhone uses.) Sending SMS messages from a conventional phone is quite painful, but doing it from the PalmOS interface is as easy as putting an appointment into your calendar. And because of the gateways between cellphone companies and the Internet, you can use the VisorPhone to send and receive brief e-mail messages. This isn’t as good as with the BlackBerry pagers that are all the rage, but it’s adequate for many purposes.

The VisorPhone can also be used as a digital GSM modem to connect to the Internet. Although this service is expensive — it costs as much as a voice call (plus the $20 per month extra my cellphone company charges) — it lets you access your e-mail, surf the Web and download the latest news with AvantGo.

The VisorPhone’s paging, e-mail and Internet features are available with the OmniSky Minstrel S wireless modem (which costs $299), and the service is $39.95 per month for unlimited use. By contrast, the VisorPhone costs $299 with cellular activation, $499 without. (My GSM provider is VoiceStream, which charges me roughly $59.99 per month for 900 minutes and 900 SMS messages.)

Because the VisorPhone requires GSM, you can only use it with digital service from companies like VoiceStream and Cingular Wireless (but only in the regions that were formerly Pac Bell) — and not with Sprint or AT&T’s cellular service. Handspring decided to build a GSM phone so that it could be used in either the U.S. or overseas with few modifications. Unfortunately, the phone isn’t dual-band or tri-band, which means that if you travel between the U.S. and Europe, you need to get two VisorPhones, one for each market. (AirPrime has announced that it is developing a CDMA module for the Visor, but that phone is not yet available.)

Because the entire PalmOS interface is at your disposal, it’s a lot easier to use a VisorPhone than a conventional cellphone. For instance, you can drag and drop the auto-dial numbers from button to button — and from page to page (the VisorPhone has five pages of auto-dial numbers). There is also an easy-to-use interface for managing the nearly 200 phone numbers you can store on the SIM chip that holds your GSM account information. And since the VisorPhone uses SIM chips, you can switch back and forth between a regular phone and the VisorPhone by swapping the chip from device to device. It’s kind of cool.

The VisorPhone isn’t the perfect blend of a PDA and a cellphone, but it comes pretty close. My biggest complaint with it is the ringer. Instead of having the traditional loud beeper common on other cellphones, the VisorPhone uses the Visor’s built-in speaker, which isn’t loud enough to be heard on a crowded street. What’s worse, the phone application comes with a limited selection of rings, and you can’t create your own tones. This is pretty disappointing, especially when my previous cellphone actually let me choose different rings for different caller-ID numbers — letting me have a different ring for my wife than for the rest of the world. With so many things done right, it is sad that Handspring got the ringer so wrong.

Another problem is battery life. The phone’s battery gives three hours of talk time or two days of standby use; SMS messaging has no noticeable impact on the battery at all. That’s not bad, but my Motorola two-way pager runs nearly five weeks on a single AA battery. Battery life will matter less after Handspring brings out its new charging dock; because of a technical oversight, the charging docks for the Visor Prism don’t charge the VisorPhone.

My final complaint is that the phone doesn’t work unless it is plugged into the Visor. This creates an interesting problem: If you are using your springboard slot for something else, like taking a photograph or playing an MP3 file, there is no way to make or receive calls. I would at least like to be able to receive calls with the VisorPhone (presumably using a headset instead of the Visor’s built-in microphone). That could be accomplished with some kind of firmware upgrade to the current phone.

Overall, I like the phone a lot. It’s not perfect, but if you’re looking to reduce technoclutter and escape from that particular geek style that demands having a different gadget in every pocket, the VisorPhone may be the best thing going.

It’s easy to understand how the article could engender such varied responses. Java is a huge industry. There are tens of thousands of companies using Java and hundreds providing tools for the language. And since there is so much disparity between the programmers that are bad at Java and those who are good at it, an attack on Java as a whole can be very threatening to those at the top of the profession. Or as one programmer with a Hotmail account told me, “Here’s this for a kicker: I make more than YOU and I’m cutting code in Java! Suck on my $200,000/yr as a Java Developer, dumb ass.”

Considering the more literate responses, the biggest criticism was that my analysis of Java was dated. “Your article is clear, concise, accurate and two years out of date,” wrote Richard Katz from Mpath Interactive. Like several other readers, Katz told me that the mainstream Java world has given up on Java-based applications — the real action is with Java application servers.

“You also ignore that the biggest growth market for Java isn’t embedded devices and the J2ME,” wrote Stu Charlton, a senior consultant at Infusion Development Corp. “It’s the enterprise, and the J2EE. (Enterprise consulting is, in fact, my profession.) The number of financial systems that push billions of dollars in transactions through Java is astounding, based on my experience of working at investment banks in Canada, the U.S. and Japan over the last three years.”

Others wrote that Java had made their lives easier by allowing them to develop Java plug-ins for Web servers on low-cost Windows NT systems, and then move the same code to $250,000 Sun Solaris servers.

In my defense, I chose to focus on desktop applications and applets because that is where most readers will encounter Java. Sun promised that Java would be the language of choice for developing desktop applications, and Sun has never repudiated this claim. It’s also instructive to look at the desktop Java experience because that is where we have the most data, and that is where it is easiest to draw conclusions.

The fact is, it’s somewhat easier to understand Java’s contribution — and its costs — when you consider server-based systems. On these systems, Java’s automatic memory management makes it a lot easier to write a reliable application server. And the plug-in nature of the Java class files makes it easier to write with systems like BEA WebLogic. But this ease comes at a price. I’ve heard of many Java installations where the server steadily uses up more memory as the day goes on, and as a result the servers need to be rebooted every night. There are other ways that performance can suffer. Some readers wrote to me that they have Java running just as fast as C; others wrote that they saw 10-fold increases in server performance when they swapped out an application written in Java for one written in C.

Readers also took me to task for my criticisms of Java portability. In the world of servers, you really can take a Java class file that was developed on a Windows NT server and move it to a Solaris box. “We develop on NT boxes or Unix boxes, and deploy on both,” wrote Geraint Preston from Lehman Brothers. “We’ve never had any problem with platform-independence, which means we can move our applications, with no recompilation, from $30,000 NT application servers to $2 million Sun E10000 servers, as and when we need.”

But think about the logic behind this argument for a moment. This argument says that it is more cost-efficient for an organization to spend $2 million on an application server and use Java than to spend $1 million on a lower-priced server and use C++ instead. It’s true, many organizations find it easier to throw hardware at a problem than to go out and hire another programmer. But it’s also easy to understand why Sun has worked so hard at instilling this attitude in the marketplace: Sun makes money selling big iron.

Indeed, take a closer look at the Java application server market, and you’ll see that there’s a contradiction of sorts. At first it seems that BEA Systems’ Tuxedo 6.x allows you to run Java plug-ins on a huge number of platforms: The company’s chart of supported operating systems includes Apple’s MacOS, Data General’s DG/UX, IBM’s AIX, and of course Microsoft Windows and Linux. But look deeper. Although the client runs on many of these servers, the server runs on considerably fewer. Java’s famed portability is there for some developers, but not for others.

Many readers took me to task for saying that Java is always interpreted — there are many just-in-time (JIT) compilers out there now. But even ignoring the fact that the bytecode still needs to be compiled every single time that it is run, and ignoring the problem that there are fundamental difficulties in getting a JIT to compile as efficiently as a traditional compiler, there are still major problems with the Java language definition that condemn it to running slower than C++ forever.

Leo Kuznetsov of Alameda, Calif., sent me an interesting example of the inherent inefficiencies of Java. Consider the case, he wrote, of appending the letters “abc” to the end of a string. In C++ concatenation is pretty simple — a buffer gets reallocated and three characters get appended to the end of the string. But in Java, three StringBuffer objects and one new String object need to be created.

David Griffiths wrote to me, “Speed is not always the issue. Time to market frequently has a bigger payoff.” This is absolutely true, and it explains the popularity of Perl, for those who want platform independence, and Microsoft’s ASP and VB Script, for those who don’t care about portability.

In that vein, several readers accused me of carrying water for Microsoft — one even insulted me by asking how much money Microsoft had paid me to place the article. These comments all demonstrate that one of the significant forces behind Java is anti-Microsoft obsession. These people hate Microsoft, and Java is the best weapon against Microsoft, so Java must be good, right? It’s a twisted logic, especially when you consider that Microsoft’s Java implementations have traditionally been better than Sun’s.

The fact is, I do plenty of cross-platform development with C++ and Qt, an application framework that’s quite efficient and easy to use. I use Qt because I don’t want the code I write to be trapped on Windows or on Unix. Likewise, I use C++ because I don’t want my code being trapped in a slow, inefficient and difficult-to-read language.

“Sir, you have hit the nail on the head with your article regarding Java performance in Salon,” wrote Brad Clawsie. “Unfortunately, the Java backers are legion. I suspect that you will get an unbearable number of flames. Java supporters are in the denial stage and they are getting a bit prickly with high-profile criticism of the language.”

It’s sad, but true. Just because Java has created an industry, just because some people are being paid more than $250,000 a year for writing Java programs, that doesn’t mean that Java is a good thing.

But such examples are exceptions rather than the rule. The vast majority of the high-profile attempts to use Java to create major desktop applications have failed. The reasons are straightforward. Java hype is built on the promulgation of two Big Lies. No. 1: Java is as fast, or faster, than other programming languages. And No. 2: Java is “portable” — it is “write-once, run-everywhere” — in other words, a Java program can be written once and then run on any kind of computer or operating system. But five years after Java’s introduction, it is still slow and cumbersome, and not only has the “write-once, run-everywhere” promise not been delivered on, it’s also turned out to not even be necessary.

Java is far from even being the first attempt at portability. Let’s not forget that the original motivation behind the C* language, way back in the early 1970s, was to create a portable computer language. The theory was that a programmer would be able to take a program written in C and be able to run it on different computers simply by recompiling* the source code. And to this end, C has been tremendously successful. I have many programs that can compile and run on Windows, on Intel-based Unix workstations, and even on Sun Ultra-SPARC servers. One of the advantages of Java over C was supposed to be that programs would be able to migrate from computer to computer without having to be recompiled. But while the portability works most of the time, Java is not, and never will be, a replacement for C or its successor C++.

The creators of Java tried to make a better C++. But they ended up with a language that is ugly, hard to read and that requires an inordinate amount of typing because of a variety of pedagogical restrictions imposed by Java’s creators. They ended up with a slow mess.

Look behind Java’s first Big Lie, and it’s clear that a well-written program in Java could never run as fast as a well-written program in C or C++. That’s because the Java bytecode is interpreted, not compiled. Programs written in C are compiled into binaries which can be executed by a specific computer processor. Programs written in Java require one more step — they must be interpreted by the Java “virtual machine” before running on a particular computer architecture. As a result, a computer running a Java program has to execute more machine-language instructions to do the same amount of work than a computer running an equivalent program written in C.

Remember back in October 1996 when Corel announced that it was creating Corel Office for Java? Corel promised us a complete rewrite of WordPerfect and other office applications, which would have supposedly allowed these new Java-based applications to run on any Java-compliant machine.

Office for Java was a failure. While the product is still in the Sun Java Solutions catalog and you can download the beta from a few archive sites on the Internet, the project was abandoned in August 1997. Download it for yourself and you’ll find out why: It’s buggy and sluggish. On my 700 MHz Pentium III, Office for Java brought memories of my old Macintosh Duo’s ill-fated attempts to run the heavily bloated Microsoft Word 6. The program’s speed, alas, was dreadful.

Netscape had similar problems when it attempted to rewrite large parts of Netscape Navigator into Java. Indeed, the damning article What Netscape Learned from Cross-Platform Software Development,” explains how Netscape’s engineers were sold on the language and started writing large chunks of Navigator into Java. Netscape was committed to delivering Navigator on nearly a dozen different platforms, so Java seemed like the perfect solution. But Netscape’s engineers couldn’t make the language perform as Sun had advertised: Java was simply too slow. Netscape’s engineers tried writing their own Java implementation, thinking that they could build a version that was faster than Sun’s, but even that didn’t work. “By mid-1998, Netscape was not only deemphasizing Java, it was even planning to replace existing implementations with C and C++.” In other words, Netscape was taking the new code that had been written in Java and was rewriting it in C and C++.

Not the best advertisement for a state-of-the-art programming language, one might argue. Still, Java’s supporters insist that many programs written in Java can be made to run as fast as programs in C. David Pollak, a San Francisco programmer who wrote an entire spreadsheet in Java and now does technology turn-arounds for companies that have Java disasters on their hands, is a huge fan of running Java applets inside Web servers: the performance difference between well-written Java and native C code is negligible, says Pollak. What really slows down Java programs, says Pollak, is when the Java applications try to display user interfaces on the screen of a computer. One way that Pollak has found to speed up Java programs is to rewrite their user interfaces in Microsoft’s Visual Basic.

Pollak is a realist. But Sun Microsystems, which originally created Java, goes further, claiming on numerous occasions ( here and here, for example) that Sun’s HotSpot technology would allow programs written in Java to actually run faster than programs in C++.

It’s true that some programs written in Java do seem to run nearly as fast as similar programs written in C. It’s also true that this experience isn’t universal. So what gives?

A 1999 study of programming efficiency goes a long way toward explaining the conflicting experiences with Java’s performance. For the study, Lutz Prechelt, a senior research associate at the school of Informatics at the University of Karlsruhe, Germany, had 38 graduate students write 40 different versions of a simple text manipulation program. The programmers, who had an average experience of eight years, created 24 versions in Java, 11 in C++, and five in C. The results, published in Communications of the ACM, were revealing. The majority of programs written in C or C++ could complete the given task in between one and five minutes. Most of the Java programs, on the other hand, required between two and 30 minutes, with some taking more than an hour. In other words, the fastest Java programs, written by the most experienced Java programmers, could significantly outperform the poorly-written C programs. But the typical Java program was much slower than the typical C program.

The upshot: It’s better to train programmers to write efficient code than to depend on new programming languages to do it for them.

“The bottom line is that if you are a skilled-enough programmer and a dedicated-enough person to wade through the docs and learn to use Java effectively, you are going to be just as effective in any of the other computer languages that people have been using in the last 30 years,” says Philip Greenspun, who teaches software engineering for Web applications at MIT. “If you weren’t good enough to program in C or LISP or PL/1 or Pascal, then you aren’t good enough to program in Java.”

Java’s second Big Lie was “write-once, run-everywhere.” As long as every Java implementation was certified, Sun told us, they would all run the same Java bytecode equally well. But as programmers became experienced with the language, they soon realized that Java was really “write-once, test-everywhere.” Every Java implementation is a little different, and sometimes those differences matter.

Write-once, run-everywhere isn’t even that useful an attribute for a programming language today. Sure, it would have been a godsend in the 1980s or the early 1990s, when a dozen different companies sold different microprocessors and different versions of Unix and other operating systems. But by 1995, when Java was introduced, the business world had pretty much settled down to use Intel-powered PCs running Windows on the desktop. The truth is that most businesses didn’t need Java’s promise of “write-once, run-everywhere.” The only people who needed it were companies like Sun and Apple: If these companies could just convince Windows programmers to code in Java, then their resulting programs would run equally well (or poorly) on Windows, Sun’s Solaris and Apple’s MacOS.

Java does have one good idea: automatic memory management, also called “garbage collection,” which actually eliminates one of the most common causes of program crashes in the world of C and C++. Of course, SmallTalk had garbage collection in 1970, and LISP had it in the 1960s — or was it the 1950s? But at least garbage collection is part of the language. (As an interesting side note, it’s fairly trivial to add garbage collection to C and C++ as well, but for some reason the practice is not popular.)

I must also admit that, despite my animosity, it looks like Java will have a bright future. But that future probably won’t be on desktop computers at all. Rather, it will be on smart cards, handheld devices and embedded computers, using the Java 2 Platform, Micro Edition. The Micro Edition actually comes with several slightly different — and mutually incompatible — versions of the Java language, but all of these Javas have essentially the same syntax and use essentially the same compilers. And since there are hundreds of thousands, if not millions, of people who have painstakingly learned Java over the past five years, companies adopting this so-called J2ME technology will have thousands of developers capable of writing programs for their products.

I’ve long argued that companies like Sony and Sharp should create developer kits for their microprocessor-containing consumer devices. With the J2ME platform, they don’t have to create the developer kit: Sun has already done the work. Security professionals have warned that this will open the door wide for hostile code and malicious programmers, but who cares? It will be cool for people to be able to download programs to run on their cell phones.

But what will be the ultimate legacy of Java? The anti-Microsoft crowd said that Java would allow Sun to finally make inroads against Microsoft’s dominance of the desktop. But in the final analysis, Java was nothing more than a ploy to capture the public’s interest and, in so doing, boost Sun’s stock price. And it worked marvelously. Java’s introduction in 1995 marked the beginning of what was essentially a five-year climb in the price of Sun’s stock: $1,000 invested in Sun on July 1995 would have been worth $18,535 at the close of trading on December 30th, 2000. Now that’s the power of Java.

McCain, R-Ariz., was clearly peeved. He said, more or less, that he didn’t need me to explain to him the purpose of the campaign finance disclosure laws. No, he wanted me to answer the question: Does selling the list of campaign contributors violate privacy?

I was testifying Tuesday morning during a meeting of the Senate Committee on Commerce, Science and Transportation. McCain had invited me to speak before the full committee on three online privacy bills that were being considered — the Consumer Internet Privacy Enhancement Act, the Consumer Privacy Protection Act and the Online Privacy Protection Act of 1999. But now he was grilling me, turning up the pressure by asking a question that seemed to demonstrate the inherent self-contradiction between my liberal democratic leanings and my pro-privacy beliefs.

The hearings were taking place inside Room 253 of the Russell Office Building. McCain was in the middle of the committee table — a huge raised desk that inscribed a majestic half-circle inside the northern side of the room. I was sitting at the witness table with three others. On my left was George Vradenburg, AOL’s senior vice president for global policy, and Scott Cooper, manager of technology policy for Hewlett-Packard. On my right was Marc Rotenberg, director of the Electronic Privacy Information Center.

When it came to our stances on privacy legislation, the four of us were split down the middle. Both Vradenburg and Cooper had spoken in favor of the Consumer Internet Privacy Enhancement Act and the Consumer Privacy Protection Act — two bills that do little more than codify today’s Internet privacy status quo. Both require only that Web sites post a privacy policy that describes what information they collect, and that companies give consumers a chance to “opt-out” or ask that their personal information not be collected.

The Consumer Internet Privacy Enhancement Act would also have the Federal Trade Commission engage the National Research Council to write another study on online privacy — a study that wouldn’t be finished for more than a year.

Along with Rotenberg, I had spoken in favor of the Online Privacy Protection Act, a bill put forth by Sen. Ernest F. Hollings, D-S.C. Really, it’s the only privacy bill of the three being considered. The Online Privacy Protection Act mandates “opt-in” — that is, it prohibits the transfer of personal information to third parties, or use of personal information for purposes other than that for which it was collected, unless the Web sites explicitly get permission from the consumer. The bill also gives consumers the right to access — that is, consumers would have a legal right to see the personal information that’s collected on them.

Now that’s a privacy bill! The Hollings bill would, furthermore, create an Office of Online Privacy within the Federal Trade Commission and give federal protection to whistleblowers within companies that violate the law — protection that’s crucial, since frequently it takes insiders to reveal egregious privacy practices.

Which is why the gentlemen from HP and AOL were so opposed to it.

The hearing came hard on the heels of a report issued by the Federal Trade Commission that called upon Congress to pass legislation protecting online privacy — a report that identified notice, choice, access and security as the key elements of any policy. But in their prepared comments, Vradenburg and Cooper repeatedly said that they supported legislation that embodied only notice and choice. Cooper, speaking for HP, said that access was simply too hard to do — what’s worse, said Vradenburg, giving consumers access to their own personal information might make the information available to hackers as well. And security was too complex to mandate in a piece of legislation.

The give and take between the senators and the representatives of the computer industry was lively. Sen. John Rockefeller, D-W.Va., chastised the executives, saying that if brokerage firms can figure out how to let people trade stocks with security, then surely a company like HP can figure out a way to let people access their own personal information. And Sen. Richard Bryan, D-Nev., criticized AOL’s insistence that people who do not “opt-out” are giving their support to the company’s marketing practices. “That’s the effect of opt-out,” he said. “Silence is acquiescence. I don’t think that most Americans see that as an effective protection.”

Indeed, most AOL users don’t like those pop-up messages that try to sell you something every time you log in, but few AOL users take the initiative to navigate through the service’s screens to turn them off. Can you imagine somebody navigating to the AOL Marketing Preferences section and clicking the button, “Yes, I do want to receive special AOL member-only pop-up offers”? It’s like sending e-mail to a spammer: “Please send me your low-interest-rate credit-card offers.” That’s why companies like AOL are in favor of opt-out, rather than opt-in.

A few minutes before the session had to end, Sen. John Kerry, D-Mass., made an impassioned speech detailing a wide variety of threats to personal privacy. It was a good speech and I agreed with many of the threats that he identified — the dangers of having medical information, banking information and even genetic information flowing over the Internet, completely out of a person’s control.

But then Kerry said that the Consumer Internet Privacy Enhancement Act did the best job of protecting consumer’s privacy! And I just couldn’t believe what he was saying. Here was Kerry, the Democratic senator from the liberal state of Massachusetts, supporting a bill that did pitifully little to protect consumer privacy. I opened my jaw, stunned. But Kerry wasn’t finished: He stated that the continued growth of the Internet depends on information services remaining free, by which he really meant “supported by advertising.” Congress should be careful with any legislation that it passes, Kerry warned, lest it kill the free Internet.

This is the same argument that Doubleclick has been making through the Network Advertising Initiative: Don’t pass privacy legislation, or you will kill the free Internet. I raised my hand and said that Americans did not have to sacrifice their privacy in order to preserve the free flow of information on the Internet. “Just because the technology makes it possible to identify a person viewing a banner advertisement doesn’t make that an effective business model.” Instead of building comprehensive profiles, all advertisers need to do is advertise consumer electronics on electronics-oriented Web sites.

And that’s when McCain asked me his question. I stopped dead in my tracks, and stumbled through my first answer. Then I took a deep breath and tried again. The crucial issue had to be the selling aspect.

“In general,” I said, “I’m opposed to private companies selling information that is collected at taxpayer expense and that the government should be making available for free in electronic form.”

After all, there are many companies that are selling this information, from court cases to patents. It’s fairly big business in Washington, and if the government did a better job making the information available in electronic form, some of the arguments about advertiser-supported Web sites might fall away.

But McCain would hear none of it. He requested again that I answer his question, and when I couldn’t do that, he went on to the next speaker.

The hearings were over in another 10 minutes or so, but McCain’s questions nagged at me for the rest of the day. Clearly, the privacy of campaign contributors is violated when their names and that information is made publicly available. But once we have made the decision to make campaign contribution information public, the next question is “how will this information be used?” My answer, that this information should not be sold by businesses, but should be given freely in electronic form by the federal government, really had nothing to do with the privacy of the contributors.

As a privacy advocate, I inevitably feel some kind of disgust whenever lists of names and personal information are sold. But is it wrong? Perhaps not. If we want this information distributed, why not have private industry do it?

The real privacy issue, I realized, has less to do with the selling of the information, and more to do with what is done with the information after it is sold:

The Human Rights Commission had actually created two databases. The first was a detailed account of threats, thefts, beatings, mutilations, murders and massacres. This database was largely created from eyewitness testimony — more than 9,000 reports in all. The second was a database that tracked the careers of El Salvador’s police and military, built largely from official records, newspaper accounts and some personal recollections.

“What we were doing was tracking them by job, rank and unit from when they graduated the military academy as young lieutenants until they retired as senior colonels or generals,” recalls Ball. “And then we crossed these two databases, by unit and time.” The technique allowed the commission to develop “statistical human rights profiles” of individual officers and units. It showed how units became more violent when certain officers took control, and cataloged the crimes that had been committed under the watch of specific individuals. Essentially, the commission had created a Who’s Who of the nastiest criminals of the country’s 20-year civil war. “And then we published them in the newspaper!”

It was a bold move for a Yankee living so far south of the border. But the move was calculated. El Salvador was in the middle of a closely watched transition from military to civilian rule. “Because it was 1992, and not 1982, they didn’t blow up our office,” says Ball. Instead, the people who had been named in the files — most of whom by then were high-ranking officials — attacked the commission in the courts. And as for Ball, he left the country.

It certainly wasn’t what Ball had expected when he signed up to work as a peace activist in El Salvador after graduating from Columbia University. His first job in Central America was as a so-called nonviolent accompaniment. “You hang around with people who were likely targets of political violence, on the premise that your witness would prevent people who wanted to do political violence from doing it,” he remembers. “It’s interesting work, but it’s actually boring when you do it. They go to meetings, but you sit around out front” and talk to the secretaries.

It was these secretaries who gave Ball his first big break. To hear him tell it, the universal experience of secretaries in offices around the world is losing files on their computers. “If you can do anything to recover their files you become a computer expert.”

As it turns out, Ball is a computer expert. He paid for his undergraduate education by working part time as a database and statistics programmer. Soon after moving to El Salvador he took a job doing computer work for the human rights office of the Lutheran Church. From there he moved to the Human Rights Commission, where he designed the databases to track El Salvador’s bloody history.

At first glance, it seems odd that the Human Rights Commission would have massive data-processing needs. Perhaps it might need a few dozen paid researchers to interview the victims and then a small team of writers to assemble the findings in a big report — but is there really a need for SQL database programmers, forms designers and multivariable analysis? Sadly, the answer is yes.

In recent years the scale of atrocities in places like Guatemala, El Salvador, Rwanda, South Africa and Kosovo has been so massive that it defies comprehension by a single person. Each of these countries has seen hundreds of thousands of victims, jointly suffering millions of individual actions and crimes. While such overwhelming brutality can simply be written up — like the volumes of testimony and records cataloged by Argentina’s Commission on Disappeared People in the Nunca Mas report — such vast amounts of data cannot be easily made sense of in a Microsoft Word document. In El Salvador, researchers figured that if they could somehow capture these events, systematize them and put them in a data bank, they could produce summary reports showing trends, propose underlying theories and motives consistent with the data and ultimately draw a comprehensive portrait of the guilty.

“I remember when I lived in El Salvador in the early ’90s, I used to go to this Saturday afternoon drinking club at which there were these human rights lawyers, some journalists, and they would tell war stories to each other,” recalls Ball. “The collective knowledge among these guys was incredible — extremely detailed knowledge of who the intelligence service worked for and who was involved — but it wasn’t systematized. When we started putting it into databases, it became incredibly useful. It could be generalized to all kinds of purposes.”

Since then, Ball has worked around the world developing software that finds hidden patterns in large databases of people’s actions. He has worked for the Truth and Reconciliation Commission in South Africa, as well as in Ethiopia, Haiti, Guatemala and the former Yugoslavia. Ironically, he uses many of the same database-mining techniques used by marketing firms to manipulate consumer opinion or by intelligence agencies to track the movements of dissidents. But in Ball’s hands, these techniques instead become tools for justice and equity.

“I think that Patrick is doing very important work for human rights by essentially professionalizing human rights, by making it more of a social science,” says Fred Abrams, a senior researcher at Human Rights Watch. “Traditionally, human rights work has been through anecdotal case studies and narrative reporting based on field research. Patrick is addressing these issues in a more scientific manner. That’s crucial. It complements the narrative reporting.”

In 1996, Ball got a full-time job at the Science and Human Rights Program at the American Association for the Advancement of Science. At the time, the association’s main foray into the field of human rights was a project that used genetic fingerprinting to match up children who had been kidnapped during Argentina’s “Dirty War” with their grandparents. Another group of scientists was in El Salvador using genetic techniques to identify remains dug up from unmarked graves. Ball met some of the association’s scientists, who quickly realized that what he was doing fit their charter of applying science to the advancement of human rights.

“Technology has leveled the playing field between human rights organizations and intelligence services,” says Ball. “Back in the ’70s, intelligence services all over the world were getting pretty impressive computer hardware. This gave them the ability to track activities, peaceful civilian activists as well as violent [individuals], in pretty precise ways, to infer patterns and to use the data analysis as the basis for oppression.”

Today the same tools can be used to build an irrefutable record that documents a history of oppression.

Ball’s work is “incredibly important,” says Harvey Weinstein, associate director of the Human Rights Center at the University of California at Berkeley. “Patrick has the capacity with this statistical knowledge to develop hard, incontrovertible statistical data to provide the kind of evidence that people need to get a good sense of the kind of human rights violations that occur in these difficult situations. He is one of the leaders in the field of trying to develop and use statistics to provide substantiation for human rights abuses.”

Over the past decade, most of the large-scale human rights databases have been built either by Ball himself or by people he personally trained. Earlier this year, Ball co-edited a book called “Making the Case,” which discusses the technical decisions that were made, and the problems that were encountered, in building these databases in El Salvador, Haiti, South Africa and Guatemala.

“After I train people, I lose them to the private sector and to government,” says Ball. “By May ’99 those projects had all ended and pretty much wrapped up,” he says. “I wanted to preserve the technology memory of how these projects happened.”

More recently, Ball has applied his statistical techniques to analyze interviews with refugees from Kosovo. In 1999, in the midst of the NATO bombing campaign, several hundred thousand people fled their homes. Although the refugees said they were fleeing Serbian militias and Serbian government forces, many on the American left claimed that the refugees were actually fleeing NATO bombs. Ball, who analyzed data from border crossing surveys of 275,000 individuals, doesn’t believe this is true.

“The core finding was that there were three phases of exodus: March 24 to April 6, when there was a huge wave. Those people almost exclusively came from the south and west. Then it starts creeping up again and peaks on April 17, and those people are all coming from north central. Then it goes down to another low point on April 24; then it comes up again in late April and early May, and those people are coming from the south.

“My conclusion was that there is a pattern here, and that pattern does not match bombing at all. There has to be some centrally coordinating cause other than bombing causing migration” — presumably, armed paramilitary groups that were traveling from village to village.

“The findings were not a surprise to me,” says Abrams of Human Rights Watch. “The Yugoslav government was claiming that people were fleeing NATO bombs, which we knew was not the case because we interviewed hundreds and hundreds of people and that is what they said. But to have the numbers reach the same conclusion was very powerful and irrefutable.”

And that’s the reason Ball has been using the tools of data mining to bolster human rights causes for so many years. The statistical techniques turn individual accounts into hard data — and data can be used to argue a cause in a public forum or in a court of law.

“The notion ‘Never forget’ is an overriding principle” of his work, says Ball. “There has been a lot of psychiatric research that shows that individual victims have a much better outcome when the truth is acknowledged. The first level of goal is truth. The second level of goal is justice; if we know what happened, maybe there is some way that the perpetrators can be punished. The third is reconciliation. The fourth level is deterrence: ‘Never again.’”

Indeed, the statistical evidence can have a lasting impact on a nation that has been through the worst of times. In Croatia, Ball says, “some of the guys who held positions in the fascist government in the ’40s now hold positions in Parliament.”

But things are different in El Salvador, at least for some of the worst offenders from the country’s troubled past. Back in 1992, after Ball and his co-workers published their list of names in the newspaper and Ball had to leave the country, military officers whose names had appeared in print sued the Human Rights Commission for defamation. So the commission went to court with computer printouts of the 9,000 testimonies and presented them as depositions. It showed the court the statistics that it had used and its methodology, and asked to subpoena the army’s own records to confirm its allegations. “The officers withdrew,” says Ball. “They didn’t think that our methods were good enough, because they thought, ‘There is no way these guys can know these things.’” But the officers were wrong.

After the court hearing, the Human Rights Commission turned over its records to the so-called Ad Hoc Commission that was overseeing the country’s transition to civilian rule. One of the jobs of the Ad Hoc Commission was to come up with a list of people who would be barred from holding public office. The people on the Ad Hoc Commission’s list matched the list that had been supplied by the HRC.

Call it the triumph of the database.