Simson Garfinkel

Lately I’ve been caught somewhere in the middle of this great battle. As a technology columnist for the Boston Globe, I’ve been routinely peppered by questions from readers. They want to know which is faster today — cable or DSL — and which system will be faster in a few years, once everybody else in their neighborhood is online. They’re also nervous about the security implications of having their computer constantly connected to the Internet and the supposed “party line” nature of cable modems.

At the same time, as the co-owner of an Internet service provider on Martha’s Vineyard, I’m nervously watching the movements of the island’s cable provider, wondering if those sleek, high-speed Internet appliances might eventually put my ISP out of business — or force us to do something entirely new. My ISP, Vineyard.net, has spent thousands of dollars deploying its own DSL system—with very mixed results.

I’ve been engaged in the debate between the proponents of cable modems and DSL for nearly a year now, and I’ve become increasingly frustrated. Both sides seem to be speaking past each other, arguing half-truths and twisting the facts to suit their own arguments. This isn’t entirely surprising, considering the amount of money on the table. But it isn’t necessary, because both technologies have merits and should be able to stand on their own.

People who don’t understand why cable modems are such a big deal have never used a cable modem. More precisely, these people have never had a high-speed, dedicated connection to the Internet. A dedicated connection blurs the boundary between where your computer stops and where the Internet starts. It’s not just that Web pages come up instantly and e-mail downloads in a flash — it changes the way you use the Net.

I don’t crack the dictionary on my bookshelf anymore. With my cable modem, it’s faster to fire up a Web browser and hit Merriam-Webster’s Dictionary site. I’ve also stopped looking at the atlas that I carry around in my car. Instead, I hit an Internet map site and print out turn-by-turn directions to my destination before I leave home.

But not everyone has embraced cable modems with the same fervor. Lots of my readers are concerned about the performance, security and reliability of cable modems.

“Your available bandwidth decreases when the other users in your neighborhood are online. Have you had any problem with this?” asked one reader. Another wrote from California: “Some people here complain that because the bandwidth is shared with everyone on the block, and because cable modems are very popular in the Bay Area, that they don’t have enough bandwidth … Compared to DSL the speeds are not all that great.”

Others are very concerned about the privacy of their personal information and the security of their home computers. A woman who works for a public relations firm in Silicon Valley shared this story: “My associate recently installed a cable modem in her home and was shocked to find that ‘Network Neighborhood’ was, literally, her neighborhood! She could see the desktops of all her connected neighbors. This seems like an enormous oversight on the part of cable modem companies, or maybe they just don’t care (more likely the latter.)” Lots of readers want to know if it’s possible for an attacker to break into their computers and access confidential files or plant viruses.

When it comes to reliability, people voice justifiable concern that their local cable company doesn’t offer a high-quality television server, so it seems even more doubtful that they will provide high-quality Internet service. After all, wrote Sang Yul Shin, a reader of mine in Boston, “Cable companies tend to lag in technical expertise and service.”

These fears about renegade cable modems have provided phone companies and Internet providers with noisy ammunition for their PR battle. This July, the New York Times ran an article “High-Speed Lines Leave Door Ajar for Hackers,” discussing the security pitfalls of leaving a computer constantly connected to the Internet. A few days later Amy McIntosh, president of Bell Atlantic Consumer Data Services, wrote a letter to the editor saying the article “incorrectly implies that home computer users with ‘always on’ Internet connections are as vulnerable to attacks from hackers whether they use a cable modem or DSL … DSL service is inherently more secure than using a cable modem because DSL provides a dedicated connection over your existing telephone line. A cable modem is more susceptible to hackers since it operates on a shared system, much like an old-fashioned party line.”

When I wrote an article about the launch of Bell Atlantic’s DSL service in April, I commented on the disappointing results: Bell Atlantic showed circuits that could theoretically deliver 600 Kbps and 7 Mbps. But my tests showed that both delivered roughly the same performance — about half the speed of my cable modem. The next day, I received an e-mail message from John Johnson, a company spokesman. “I thought the emphasis on milliseconds of difference in cable modem/DSL performance was misplaced,” he said. “The bigger picture, in my view, is that DSL gives users dedicated bandwidth. While you yourself may not have run into the slowdowns that other cable modem users have found at peak hours, I’m confident it’s only a matter of time before you do. When that happens, I hope you’ll describe the benefits of dedicated bandwidth to your readers.”

Internet service providers have another legitimate fear about cable modems: They offer high-speed service at a price that most ISPs can’t touch. Consider the cable modem service that I purchase from MediaOne. For $51.50 a month, I get a service that routinely delivers between 300 and 600 kilobits per second in both directions. For comparison, ISPs in Boston that are selling high-speed service are charging roughly $200 a month for 384 Kbps and $300 a month for 768 Kbps.

One of the reasons that cable service costs less than DSL is because it’s a shared facility. A coaxial cable traveling in a neighborhood from house to house can provide high-speed service to thousands of customers. A single piece of equipment at the cable company’s office can patch those thousands of customers onto the Internet. DSL, on the other hand, requires a separate pair of wires for each subscriber: The phone company needs to install a special DSL “modem” for each phone line at its central office. So high-speed Internet service delivered over cable should be less expensive than the same service delivered over a DSL link.

And cable companies are particularly eager to sign up cable modem users. Across the nation, customers have demonstrated that they are more than willing to pay between $30 and $50 per month extra for high-speed Internet access; that’s substantially more than the cost of a typical premium cable channel like Showtime or HBO. What’s more, the same two-way digital networks that allow companies like MediaOne to deliver the Internet can also be used to deliver telephone service, further increasing cable revenues. The Internet makes it possible for cable companies to double or triple their revenues — using much of their existing infrastructure.

The argument that cable modem connections are inferior to DSL connections because the bandwidth on a cable link is “shared” is disingenuous as best. The entire Internet is based upon shared bandwidth. The bandwidth on an individual subscriber’s DSL connection may not be shared; the DSL connection ultimately terminates at an Internet router, at which point the multiple streams are merged together into one or more shared upstream connections. Likewise, cross-country Internet links are shared between thousands of ISPs and millions of users. Even DSL circuits can be shared: Bell Atlantic’s DSL deployment shares a single pair of copper wires that both voice and data can use simultaneously.

The issue isn’t whether or not a link is shared. The issue is whether or not there is enough bandwidth on the shared link to satisfy all of the users at a particular time.

In bandwidth, cable is king: With the exception of fiber optics, there is no faster way to send a data stream from one location to another. In my tests, I’ve seen instantaneous speeds of 7 megabits per second over my cable modem — that’s the underlying speed of the data channel between my house and the next router on the network. Of course, I don’t actually see 7 Mbps in my daily use. When I transfer more than a few packets of data, the cable modem is programmed to slow down the rate of transfer to roughly 600 Kbps. This prevents me from taking more than my fair share of the available bandwidth.

From the cable provider’s perspective, the key to maintaining good performance is to keep a watchful eye on customers’ bandwidth utilization, then deploy more facilities in advance of customer demand. For example, by monitoring my cable modem, I’ve determined that slightly more than 800 subscribers share my cable segment. That number will increase as MediaOne sells more cable modems in my neighborhood.

Despite its capacity, at some point the cable will reach saturation. MediaOne can then reduce the congestion by splitting the cable segment in half and installing another router at its office. I suspect that the reported problems plaguing cable modems in the Bay Area are occurring because demand has grown too fast for the cable company to keep up.

That’s not the end of the bandwidth story, of course. Two factors determine just how fast the Internet will seem to the average home user. The first factor is the connection between the subscriber’s home computer and the ISP. But the second factor, the one that’s frequently overlooked, is the connection between the ISP and the rest of the Net.

MediaOne has done a spectacular job in the Boston area of connecting its cable modem subscribers with the rest of the global network. Packets take 12 milliseconds (20 hops) to make the round-trip from my desk to the Exodus hub in Pennsauken, N.J. (For comparison, a particle of light would take just 3.2 milliseconds to make the round-trip, implying that MediaOne has installed some awesomely fast routers between Cambridge and the Garden State.) I am 80 milliseconds away from Microsoft in Redmond, Wash. I can download Netscape Communicator 4.5 — which resides on a server in San Jose — in less than 4 minutes.

But I’m even more impressed by the way that MediaOne has worked to deploy connectivity within my metropolitan area. A high-speed connection between MediaOne and the Massachusetts Institute of Technology puts just 3 milliseconds between my desk and the Media Lab. That link is so tremendously fast that I can access machines at MIT about as easily as computers on my local area network. I can download and install the entire FreeBSD operating system from MIT in less than 10 minutes.

To gauge Bell Atlantic’s DSL Internet service, I called up the company and had its InfoSpeed service installed at my house in August. The company sent me a DSL modem in the mail, and added the DSL service to my existing phone line. Bell Atlantic also sent me a dozen “microfilters” that I needed to install on the other telephones in my house. Without the microfilters in place, the DSL connection’s high-speed data stream sounds like a loud hiss in every phone. (Truth be told, though, even with the microfilters in place I could still hear the DSL on some of my phones.)

The performance was disappointing. But the fault was not with the DSL technology, but with Bell Atlantic’s Internet service provider, BellAtlantic.net, which provides the connection between the DSL line and the Internet. Bell Atlantic, it seemed, didn’t have very good connectivity in the Boston area. When I tried to connect to MIT, my Internet packets ended up going through New York or northern Virginia — and sometimes both places. It took twice as long to download Netscape Navigator.

Now, to be fair, you can get DSL from companies other than Bell Atlantic. At my office in Boston, for instance, we have a 384K DSL connection from Concentric. It too has been something of a disappointment, but for a different reason. Concentric has been advertising heavily in the Boston subway system, but the company is fundamentally a West Coast ISP with delusions of grandeur. Specifically, Concentric doesn’t actually have any employees in Boston. Instead, it contracts with a company called Covad to do all the grunt work.

Concentric’s West Coast bias became clear when I looked at packet traces between my Boston office and the rest of the Internet. We have truly spectacular connectivity between our office and downtown San Jose — we’re just 40 milliseconds from MAE West. This speed is a good thing, because packets from our office destined for local sites, frequently go out to California, or Chicago or New York, before looping back to Beantown. Concentric, like Bell Atlantic, doesn’t seem to be a peer with the major Boston Internet service providers or with the universities.

I have mentioned, somewhat offhandedly, that I had surveyed my cable modem and discovered that I was sharing my channel with roughly 800 other customers. If you are concerned about your electronic privacy, that statement should give you some cause for alarm. How is it possible that I could determine this information? And what else can I learn about my network neighborhood?

The answers to these questions are, alas, quite technical and vary greatly between one cable system and another. Some of today’s cable modem networks are quite secure, while others are wide open.

Cable modem systems are roughly based on the Ethernet technology developed at Xerox PARC in the 1970s and popularized in the late ’80s and early ’90s. To the first approximation, a typical neighborhood cable network looks like a big local area network.

This structure also lowers the cost of deployment. Making a neighborhood network look like one big Ethernet saved a bundle of money for the cable modem companies, because it let them use off-the-shelf hardware and software. Practically every operating system in use today can speak the Internet’s TCP/IP protocols over Ethernet. By making the cable modems look like little Ethernet hubs, vendors could avoid writing network drivers for Windows, MacOS, Linux — and every other operating system, for that matter. Mimicking Ethernet also let cable companies use off-the-shelf Ethernet cards in desktop and laptop computers, further cutting costs. Overall, going with Ethernet was the right decision.

Unfortunately, Ethernet has a problem that makes it somewhat unsuited to a neighborhood environment: It’s a broadcast network. On a typical office LAN, for example, you can run a program called a “network analyzer” and see the packets of every other machine. The computer underground has written programs called “packet sniffers,” which exploit this feature to capture a victim’s password as it is typed.

When the first cable modem networks were deployed in the Boston area, I loaded my favorite packet sniffer onto a floppy disk and headed over to my friend Richard’s house. Richard had already told me that he could see other Macintosh computers in his neighborhood, using his Mac’s “Chooser” program. Sometimes people even accidentally printed files on his printer. With packet sniffer in hand, I was prepared to capture the passwords and Web traffic of everybody else in the neighborhood. Then I planned to write an article about the total lack of privacy on cable modem networks

But when I got to Richard’s house and set up the sniffer, something went wrong: I could see only a tiny fraction of the network’s traffic. The engineers at the company that had built my friend’s cable modem weren’t bozos after all. Each cable modem on the network had been programmed to filter out packets destined for other subscribers. To use the jargon of Ethernet, the cable modems weren’t acting as Ethernet hubs, but rather as “bridges” or “switches.” Richard could still use his Mac’s Chooser to see the other computers because AppleTalk’s network protocol uses broadcast Ethernet packets to let the machines on the network find each other.

When a computer on an Ethernet network sends a broadcast packet, that packet is automatically received by every other host on the network. Microsoft Windows uses broadcast packets to find the hosts for the “Network Neighborhood” window. The Internet’s “ARP” protocol uses Ethernet broadcast to determine which computer on a local area network has a particular address.

Cable modems transmitting Ethernet broadcast packets to every subscriber on the neighborhood are a significant vulnerability, easily exploited by a technically savvy attacker. For example, using a freely available program called “arpwatch,” I can scan for the ARP packets and detect how many subscribers are on my cable segment. Since MediaOne has assigned host names that look a lot like user names (e.g. sjones.ne.mediaone.net), I can learn the names of my cyber-neighbors. I can also learn when the ARP packets are sent, and establish when my neighbors are using their computers — and when they are at work.

Cable modems have another security risk. But that risk doesn’t have anything to do with the modems themselves; rather, it arises from the Windows operating system. That vulnerability is called file sharing. If you have two computers in the same house, you can turn on file sharing to let one computer access the other’s hard drive. The danger here is that many people turn on file sharing but don’t set up a password. Put a computer configured this way on the Internet, and a hundred million people can look at any of the files on your machine.

Different cable companies have tried different approaches to address these security issues. MediaOne, for example, blocks the particular TCP/IP ports used by the Microsoft file sharing protocol. (The company can unblock the ports on a subscriber-by-subscriber basis, if you call them up and make the request.) Other companies leave the ports open, and warn their customers to disable file sharing. I prefer MediaOne’s approach.

The ARP problem, meanwhile, will be solved by the next-generation cable modems that implement the so-called DOCSIS 1.1 protocol. Instead of broadcasting ARP packets over the entire cable segment, DOCSIS 1.1 makes sure that each customer will only see the ARP messages intended for his or her machine. As an added protection, DOCSIS 1.1 is capable of encrypting all information sent over the cable itself, with a separate encryption key for each customer. This security measure prevents an attacker from splicing their own cable modem into the backbone, the way that some people used to hook up unauthorized cable decoders to get free cable TV service.

DSL users have pretty much the same set of security concerns as cable modem customers. That’s because DSL modems can be set up as routers or bridges. At my Boston office, the Concentric DSL modem is set up as a router. Concentric has given us 6 IP addresses for our own use, and we can’t see anybody else’s traffic. The Bell Atlantic DSL modem, on the other hand, is configured as a bridge, with potentially 253 other customers sharing the same network — in this case, a VLAN, or Virtual Local Area Network.

Neither Concentric nor Bell Atlantic filter their DSL connections, potentially leaving customers open to file-sharing attacks. The service technician who installed the Concentric modem told me that I could have it configured as a firewall, if I wanted. Bell Atlantic, on the other hand, simply gave me a small pamphlet that explained the dangers of the Internet and suggested that I turn off file sharing on my Windows computer. As for the problem with broadcast ARPs, Bell Atlantic has its own, somewhat inelegant solution. Instead of filtering the broadcast packets, Bell Atlantic has simply programmed its computers to make it impossible for me to exchange packets with any of the 253 other subscribers on my local area network. Most of the time this shouldn’t cause any problem, but if two customers want to play Doom with each other over the network, and they happen to be on the same VLAN, they’re out of luck.

To the future

Whether or not cable modems will be more or less reliable than DSL is an open question. When it comes to delivering a consistently reliable service, cable companies certainly do not have a good history. This may be because they’ve never been forced to deliver a consistently high-quality service: Although it might be an inconvenience to go without cable for a few nights, nobody will die if they can’t get their fill of “Star Trek” and “Gilligan’s Island.” Telephone companies, however, have been required for years to deliver highly reliable dial tone, since 911 is the basis of police, fire and ambulance services. As cable companies begin to provide their own dial tone, they will fall under the same regulations.

With that said, my experience has been the reverse. During the past year, my cable modem has been down only a single day. Meanwhile, I have lost dial tone on my primary phone line on no less than six occasions. Bell Atlantic says this is because I live in an old neighborhood where the wires aren’t in very good shape; on the other hand, the cable was installed relatively recently.

If the history of technology is any guide, however, it’s unlikely that the battle between cable modems and DSL will be won on technical merits. It’s exceedingly difficult to find a single case within the past 50 years where a better technology won out in the marketplace against an inferior one. Beta lost to VHS, after all, and the sleek “RISC” microprocessors from companies like Sun Microsystems and MIPS lost out to Intel’s technologically inferior Pentium chips. Ultimately, this battle will be won and lost on mundane issues like price and quality of service.

It’s for these reasons, in fact, that I firmly believe companies like MediaOne should be forced to open up their cable networks to other providers. Gargantuan companies like Bell Atlantic ultimately will be able to compete against the cable modem providers: They’ll just spend a lot of money to make their DSL offerings competitive with the cable systems. But given all of the natural advantages that cable enjoys, small-to-medium-sized ISPs really don’t stand a chance. Unless these networks are opened for all to use, consumers may soon have just two choices for an Internet provider: their cable company or their Baby Bell.

The combination of the Visor PDA and phone does a better job than all of these other attempts — it’s even better than the ill-fated pdQ. And in a curious twist, the Visor duo makes a more usable cellphone than most stand-alone cellphones. It’s a real example of integration creating a better product — something that the high-tech industry constantly promises, but rarely delivers.

The original Handspring Visor was designed with the VisorPhone in mind. Back when the folks at Handspring (incidentally, the same people who created the original Palm organizer) were designing the first Visors, they wanted to create a phone module but simply didn’t have the time or the engineering resources. So they did the next best thing — they designed into the Visor the connectors and additional hardware that the future phone module would require. As a result, the first Visors to come off the line had a little hole in the lower left-hand corner, with a tiny microphone wired in place. But the microphone wasn’t wired to anything inside the Visor itself; instead, a pair of wires snakes from the microphone to the Visor’s “springboard” expansion slot. When you plug the VisorPhone module into the Visor, it uses the built-in mike — which sounds great.

Likewise, a second pair of wires goes from the connector at the base of the Visor to the springboard and is used for charging the phone’s batteries (which requires much more power than can be provided over a standard Universal Serial Bus interface). The speaker for the VisorPhone is in the black module itself. (There is also a jack for a headset.)

Because of this advance planning, the Visor and VisorPhone combo is significantly smaller than other ill-fated integration attempts. My pdQ was a big brick that was awkward to carry; worse, the pdQ had two different user interfaces — a keypad for the phone and the LCD screen for the Palm. The new combo, on the other hand, weighs 8.5 ounces and fits in my shirt pocket. Better still, the VisorPhone is completely controlled by a set of PalmOS applications that automatically get installed on the Visor when the module is inserted.

Making phone calls is pretty easy. You start by clicking the “phone” button on top of the phone module, which makes the Visor display an application with 10 auto-dial buttons. Touch a button with the Visor stylus (or your finger) and the phone dials. If the phone number you want to call isn’t in your auto-dialer, you click the “phone” button again and the Visor will display a telephone keypad; you tap the number and then talk. You can also dial any phone number in the Palm’s address book by tapping the number displayed and then confirming that you wish to dial it. Finally, you can dial any number in a memo or on your calendar by selecting the number with the stylus and then pressing the “dial” button.

The Visor displays the call’s elapsed time, and you can put a person on hold by tapping a button on the phone application. You can also place a second call or handle a second incoming call. Or, if you are exceptionally good at multitasking, you can click into another PalmOS application. You can even turn off the Palm, since the phone has its own battery and built-in microcomputer. Just about the only thing you can’t do is pull the VisorPhone out of the Visor’s springboard slot — that makes it hang up.

But voice calls are just the beginning of the action. The second button at the top of the module is for sending and receiving SMS messages. (SMS stands for “short messaging service,” and is built into the GSM the VisorPhone uses.) Sending SMS messages from a conventional phone is quite painful, but doing it from the PalmOS interface is as easy as putting an appointment into your calendar. And because of the gateways between cellphone companies and the Internet, you can use the VisorPhone to send and receive brief e-mail messages. This isn’t as good as with the BlackBerry pagers that are all the rage, but it’s adequate for many purposes.

The VisorPhone can also be used as a digital GSM modem to connect to the Internet. Although this service is expensive — it costs as much as a voice call (plus the $20 per month extra my cellphone company charges) — it lets you access your e-mail, surf the Web and download the latest news with AvantGo.

The VisorPhone’s paging, e-mail and Internet features are available with the OmniSky Minstrel S wireless modem (which costs $299), and the service is $39.95 per month for unlimited use. By contrast, the VisorPhone costs $299 with cellular activation, $499 without. (My GSM provider is VoiceStream, which charges me roughly $59.99 per month for 900 minutes and 900 SMS messages.)

Because the VisorPhone requires GSM, you can only use it with digital service from companies like VoiceStream and Cingular Wireless (but only in the regions that were formerly Pac Bell) — and not with Sprint or AT&T’s cellular service. Handspring decided to build a GSM phone so that it could be used in either the U.S. or overseas with few modifications. Unfortunately, the phone isn’t dual-band or tri-band, which means that if you travel between the U.S. and Europe, you need to get two VisorPhones, one for each market. (AirPrime has announced that it is developing a CDMA module for the Visor, but that phone is not yet available.)

Because the entire PalmOS interface is at your disposal, it’s a lot easier to use a VisorPhone than a conventional cellphone. For instance, you can drag and drop the auto-dial numbers from button to button — and from page to page (the VisorPhone has five pages of auto-dial numbers). There is also an easy-to-use interface for managing the nearly 200 phone numbers you can store on the SIM chip that holds your GSM account information. And since the VisorPhone uses SIM chips, you can switch back and forth between a regular phone and the VisorPhone by swapping the chip from device to device. It’s kind of cool.

The VisorPhone isn’t the perfect blend of a PDA and a cellphone, but it comes pretty close. My biggest complaint with it is the ringer. Instead of having the traditional loud beeper common on other cellphones, the VisorPhone uses the Visor’s built-in speaker, which isn’t loud enough to be heard on a crowded street. What’s worse, the phone application comes with a limited selection of rings, and you can’t create your own tones. This is pretty disappointing, especially when my previous cellphone actually let me choose different rings for different caller-ID numbers — letting me have a different ring for my wife than for the rest of the world. With so many things done right, it is sad that Handspring got the ringer so wrong.

Another problem is battery life. The phone’s battery gives three hours of talk time or two days of standby use; SMS messaging has no noticeable impact on the battery at all. That’s not bad, but my Motorola two-way pager runs nearly five weeks on a single AA battery. Battery life will matter less after Handspring brings out its new charging dock; because of a technical oversight, the charging docks for the Visor Prism don’t charge the VisorPhone.

My final complaint is that the phone doesn’t work unless it is plugged into the Visor. This creates an interesting problem: If you are using your springboard slot for something else, like taking a photograph or playing an MP3 file, there is no way to make or receive calls. I would at least like to be able to receive calls with the VisorPhone (presumably using a headset instead of the Visor’s built-in microphone). That could be accomplished with some kind of firmware upgrade to the current phone.

Overall, I like the phone a lot. It’s not perfect, but if you’re looking to reduce technoclutter and escape from that particular geek style that demands having a different gadget in every pocket, the VisorPhone may be the best thing going.

It’s easy to understand how the article could engender such varied responses. Java is a huge industry. There are tens of thousands of companies using Java and hundreds providing tools for the language. And since there is so much disparity between the programmers that are bad at Java and those who are good at it, an attack on Java as a whole can be very threatening to those at the top of the profession. Or as one programmer with a Hotmail account told me, “Here’s this for a kicker: I make more than YOU and I’m cutting code in Java! Suck on my $200,000/yr as a Java Developer, dumb ass.”

Considering the more literate responses, the biggest criticism was that my analysis of Java was dated. “Your article is clear, concise, accurate and two years out of date,” wrote Richard Katz from Mpath Interactive. Like several other readers, Katz told me that the mainstream Java world has given up on Java-based applications — the real action is with Java application servers.

“You also ignore that the biggest growth market for Java isn’t embedded devices and the J2ME,” wrote Stu Charlton, a senior consultant at Infusion Development Corp. “It’s the enterprise, and the J2EE. (Enterprise consulting is, in fact, my profession.) The number of financial systems that push billions of dollars in transactions through Java is astounding, based on my experience of working at investment banks in Canada, the U.S. and Japan over the last three years.”

Others wrote that Java had made their lives easier by allowing them to develop Java plug-ins for Web servers on low-cost Windows NT systems, and then move the same code to $250,000 Sun Solaris servers.

In my defense, I chose to focus on desktop applications and applets because that is where most readers will encounter Java. Sun promised that Java would be the language of choice for developing desktop applications, and Sun has never repudiated this claim. It’s also instructive to look at the desktop Java experience because that is where we have the most data, and that is where it is easiest to draw conclusions.

The fact is, it’s somewhat easier to understand Java’s contribution — and its costs — when you consider server-based systems. On these systems, Java’s automatic memory management makes it a lot easier to write a reliable application server. And the plug-in nature of the Java class files makes it easier to write with systems like BEA WebLogic. But this ease comes at a price. I’ve heard of many Java installations where the server steadily uses up more memory as the day goes on, and as a result the servers need to be rebooted every night. There are other ways that performance can suffer. Some readers wrote to me that they have Java running just as fast as C; others wrote that they saw 10-fold increases in server performance when they swapped out an application written in Java for one written in C.

Readers also took me to task for my criticisms of Java portability. In the world of servers, you really can take a Java class file that was developed on a Windows NT server and move it to a Solaris box. “We develop on NT boxes or Unix boxes, and deploy on both,” wrote Geraint Preston from Lehman Brothers. “We’ve never had any problem with platform-independence, which means we can move our applications, with no recompilation, from $30,000 NT application servers to $2 million Sun E10000 servers, as and when we need.”

But think about the logic behind this argument for a moment. This argument says that it is more cost-efficient for an organization to spend $2 million on an application server and use Java than to spend $1 million on a lower-priced server and use C++ instead. It’s true, many organizations find it easier to throw hardware at a problem than to go out and hire another programmer. But it’s also easy to understand why Sun has worked so hard at instilling this attitude in the marketplace: Sun makes money selling big iron.

Indeed, take a closer look at the Java application server market, and you’ll see that there’s a contradiction of sorts. At first it seems that BEA Systems’ Tuxedo 6.x allows you to run Java plug-ins on a huge number of platforms: The company’s chart of supported operating systems includes Apple’s MacOS, Data General’s DG/UX, IBM’s AIX, and of course Microsoft Windows and Linux. But look deeper. Although the client runs on many of these servers, the server runs on considerably fewer. Java’s famed portability is there for some developers, but not for others.

Many readers took me to task for saying that Java is always interpreted — there are many just-in-time (JIT) compilers out there now. But even ignoring the fact that the bytecode still needs to be compiled every single time that it is run, and ignoring the problem that there are fundamental difficulties in getting a JIT to compile as efficiently as a traditional compiler, there are still major problems with the Java language definition that condemn it to running slower than C++ forever.

Leo Kuznetsov of Alameda, Calif., sent me an interesting example of the inherent inefficiencies of Java. Consider the case, he wrote, of appending the letters “abc” to the end of a string. In C++ concatenation is pretty simple — a buffer gets reallocated and three characters get appended to the end of the string. But in Java, three StringBuffer objects and one new String object need to be created.

David Griffiths wrote to me, “Speed is not always the issue. Time to market frequently has a bigger payoff.” This is absolutely true, and it explains the popularity of Perl, for those who want platform independence, and Microsoft’s ASP and VB Script, for those who don’t care about portability.

In that vein, several readers accused me of carrying water for Microsoft — one even insulted me by asking how much money Microsoft had paid me to place the article. These comments all demonstrate that one of the significant forces behind Java is anti-Microsoft obsession. These people hate Microsoft, and Java is the best weapon against Microsoft, so Java must be good, right? It’s a twisted logic, especially when you consider that Microsoft’s Java implementations have traditionally been better than Sun’s.

The fact is, I do plenty of cross-platform development with C++ and Qt, an application framework that’s quite efficient and easy to use. I use Qt because I don’t want the code I write to be trapped on Windows or on Unix. Likewise, I use C++ because I don’t want my code being trapped in a slow, inefficient and difficult-to-read language.

“Sir, you have hit the nail on the head with your article regarding Java performance in Salon,” wrote Brad Clawsie. “Unfortunately, the Java backers are legion. I suspect that you will get an unbearable number of flames. Java supporters are in the denial stage and they are getting a bit prickly with high-profile criticism of the language.”

It’s sad, but true. Just because Java has created an industry, just because some people are being paid more than $250,000 a year for writing Java programs, that doesn’t mean that Java is a good thing.

But such examples are exceptions rather than the rule. The vast majority of the high-profile attempts to use Java to create major desktop applications have failed. The reasons are straightforward. Java hype is built on the promulgation of two Big Lies. No. 1: Java is as fast, or faster, than other programming languages. And No. 2: Java is “portable” — it is “write-once, run-everywhere” — in other words, a Java program can be written once and then run on any kind of computer or operating system. But five years after Java’s introduction, it is still slow and cumbersome, and not only has the “write-once, run-everywhere” promise not been delivered on, it’s also turned out to not even be necessary.

Java is far from even being the first attempt at portability. Let’s not forget that the original motivation behind the C* language, way back in the early 1970s, was to create a portable computer language. The theory was that a programmer would be able to take a program written in C and be able to run it on different computers simply by recompiling* the source code. And to this end, C has been tremendously successful. I have many programs that can compile and run on Windows, on Intel-based Unix workstations, and even on Sun Ultra-SPARC servers. One of the advantages of Java over C was supposed to be that programs would be able to migrate from computer to computer without having to be recompiled. But while the portability works most of the time, Java is not, and never will be, a replacement for C or its successor C++.

The creators of Java tried to make a better C++. But they ended up with a language that is ugly, hard to read and that requires an inordinate amount of typing because of a variety of pedagogical restrictions imposed by Java’s creators. They ended up with a slow mess.

Look behind Java’s first Big Lie, and it’s clear that a well-written program in Java could never run as fast as a well-written program in C or C++. That’s because the Java bytecode is interpreted, not compiled. Programs written in C are compiled into binaries which can be executed by a specific computer processor. Programs written in Java require one more step — they must be interpreted by the Java “virtual machine” before running on a particular computer architecture. As a result, a computer running a Java program has to execute more machine-language instructions to do the same amount of work than a computer running an equivalent program written in C.

Remember back in October 1996 when Corel announced that it was creating Corel Office for Java? Corel promised us a complete rewrite of WordPerfect and other office applications, which would have supposedly allowed these new Java-based applications to run on any Java-compliant machine.

Office for Java was a failure. While the product is still in the Sun Java Solutions catalog and you can download the beta from a few archive sites on the Internet, the project was abandoned in August 1997. Download it for yourself and you’ll find out why: It’s buggy and sluggish. On my 700 MHz Pentium III, Office for Java brought memories of my old Macintosh Duo’s ill-fated attempts to run the heavily bloated Microsoft Word 6. The program’s speed, alas, was dreadful.

Netscape had similar problems when it attempted to rewrite large parts of Netscape Navigator into Java. Indeed, the damning article What Netscape Learned from Cross-Platform Software Development,” explains how Netscape’s engineers were sold on the language and started writing large chunks of Navigator into Java. Netscape was committed to delivering Navigator on nearly a dozen different platforms, so Java seemed like the perfect solution. But Netscape’s engineers couldn’t make the language perform as Sun had advertised: Java was simply too slow. Netscape’s engineers tried writing their own Java implementation, thinking that they could build a version that was faster than Sun’s, but even that didn’t work. “By mid-1998, Netscape was not only deemphasizing Java, it was even planning to replace existing implementations with C and C++.” In other words, Netscape was taking the new code that had been written in Java and was rewriting it in C and C++.

Not the best advertisement for a state-of-the-art programming language, one might argue. Still, Java’s supporters insist that many programs written in Java can be made to run as fast as programs in C. David Pollak, a San Francisco programmer who wrote an entire spreadsheet in Java and now does technology turn-arounds for companies that have Java disasters on their hands, is a huge fan of running Java applets inside Web servers: the performance difference between well-written Java and native C code is negligible, says Pollak. What really slows down Java programs, says Pollak, is when the Java applications try to display user interfaces on the screen of a computer. One way that Pollak has found to speed up Java programs is to rewrite their user interfaces in Microsoft’s Visual Basic.

Pollak is a realist. But Sun Microsystems, which originally created Java, goes further, claiming on numerous occasions ( here and here, for example) that Sun’s HotSpot technology would allow programs written in Java to actually run faster than programs in C++.

It’s true that some programs written in Java do seem to run nearly as fast as similar programs written in C. It’s also true that this experience isn’t universal. So what gives?

A 1999 study of programming efficiency goes a long way toward explaining the conflicting experiences with Java’s performance. For the study, Lutz Prechelt, a senior research associate at the school of Informatics at the University of Karlsruhe, Germany, had 38 graduate students write 40 different versions of a simple text manipulation program. The programmers, who had an average experience of eight years, created 24 versions in Java, 11 in C++, and five in C. The results, published in Communications of the ACM, were revealing. The majority of programs written in C or C++ could complete the given task in between one and five minutes. Most of the Java programs, on the other hand, required between two and 30 minutes, with some taking more than an hour. In other words, the fastest Java programs, written by the most experienced Java programmers, could significantly outperform the poorly-written C programs. But the typical Java program was much slower than the typical C program.

The upshot: It’s better to train programmers to write efficient code than to depend on new programming languages to do it for them.

“The bottom line is that if you are a skilled-enough programmer and a dedicated-enough person to wade through the docs and learn to use Java effectively, you are going to be just as effective in any of the other computer languages that people have been using in the last 30 years,” says Philip Greenspun, who teaches software engineering for Web applications at MIT. “If you weren’t good enough to program in C or LISP or PL/1 or Pascal, then you aren’t good enough to program in Java.”

Java’s second Big Lie was “write-once, run-everywhere.” As long as every Java implementation was certified, Sun told us, they would all run the same Java bytecode equally well. But as programmers became experienced with the language, they soon realized that Java was really “write-once, test-everywhere.” Every Java implementation is a little different, and sometimes those differences matter.

Write-once, run-everywhere isn’t even that useful an attribute for a programming language today. Sure, it would have been a godsend in the 1980s or the early 1990s, when a dozen different companies sold different microprocessors and different versions of Unix and other operating systems. But by 1995, when Java was introduced, the business world had pretty much settled down to use Intel-powered PCs running Windows on the desktop. The truth is that most businesses didn’t need Java’s promise of “write-once, run-everywhere.” The only people who needed it were companies like Sun and Apple: If these companies could just convince Windows programmers to code in Java, then their resulting programs would run equally well (or poorly) on Windows, Sun’s Solaris and Apple’s MacOS.

Java does have one good idea: automatic memory management, also called “garbage collection,” which actually eliminates one of the most common causes of program crashes in the world of C and C++. Of course, SmallTalk had garbage collection in 1970, and LISP had it in the 1960s — or was it the 1950s? But at least garbage collection is part of the language. (As an interesting side note, it’s fairly trivial to add garbage collection to C and C++ as well, but for some reason the practice is not popular.)

I must also admit that, despite my animosity, it looks like Java will have a bright future. But that future probably won’t be on desktop computers at all. Rather, it will be on smart cards, handheld devices and embedded computers, using the Java 2 Platform, Micro Edition. The Micro Edition actually comes with several slightly different — and mutually incompatible — versions of the Java language, but all of these Javas have essentially the same syntax and use essentially the same compilers. And since there are hundreds of thousands, if not millions, of people who have painstakingly learned Java over the past five years, companies adopting this so-called J2ME technology will have thousands of developers capable of writing programs for their products.

I’ve long argued that companies like Sony and Sharp should create developer kits for their microprocessor-containing consumer devices. With the J2ME platform, they don’t have to create the developer kit: Sun has already done the work. Security professionals have warned that this will open the door wide for hostile code and malicious programmers, but who cares? It will be cool for people to be able to download programs to run on their cell phones.

But what will be the ultimate legacy of Java? The anti-Microsoft crowd said that Java would allow Sun to finally make inroads against Microsoft’s dominance of the desktop. But in the final analysis, Java was nothing more than a ploy to capture the public’s interest and, in so doing, boost Sun’s stock price. And it worked marvelously. Java’s introduction in 1995 marked the beginning of what was essentially a five-year climb in the price of Sun’s stock: $1,000 invested in Sun on July 1995 would have been worth $18,535 at the close of trading on December 30th, 2000. Now that’s the power of Java.

McCain, R-Ariz., was clearly peeved. He said, more or less, that he didn’t need me to explain to him the purpose of the campaign finance disclosure laws. No, he wanted me to answer the question: Does selling the list of campaign contributors violate privacy?

I was testifying Tuesday morning during a meeting of the Senate Committee on Commerce, Science and Transportation. McCain had invited me to speak before the full committee on three online privacy bills that were being considered — the Consumer Internet Privacy Enhancement Act, the Consumer Privacy Protection Act and the Online Privacy Protection Act of 1999. But now he was grilling me, turning up the pressure by asking a question that seemed to demonstrate the inherent self-contradiction between my liberal democratic leanings and my pro-privacy beliefs.

The hearings were taking place inside Room 253 of the Russell Office Building. McCain was in the middle of the committee table — a huge raised desk that inscribed a majestic half-circle inside the northern side of the room. I was sitting at the witness table with three others. On my left was George Vradenburg, AOL’s senior vice president for global policy, and Scott Cooper, manager of technology policy for Hewlett-Packard. On my right was Marc Rotenberg, director of the Electronic Privacy Information Center.

When it came to our stances on privacy legislation, the four of us were split down the middle. Both Vradenburg and Cooper had spoken in favor of the Consumer Internet Privacy Enhancement Act and the Consumer Privacy Protection Act — two bills that do little more than codify today’s Internet privacy status quo. Both require only that Web sites post a privacy policy that describes what information they collect, and that companies give consumers a chance to “opt-out” or ask that their personal information not be collected.

The Consumer Internet Privacy Enhancement Act would also have the Federal Trade Commission engage the National Research Council to write another study on online privacy — a study that wouldn’t be finished for more than a year.

Along with Rotenberg, I had spoken in favor of the Online Privacy Protection Act, a bill put forth by Sen. Ernest F. Hollings, D-S.C. Really, it’s the only privacy bill of the three being considered. The Online Privacy Protection Act mandates “opt-in” — that is, it prohibits the transfer of personal information to third parties, or use of personal information for purposes other than that for which it was collected, unless the Web sites explicitly get permission from the consumer. The bill also gives consumers the right to access — that is, consumers would have a legal right to see the personal information that’s collected on them.

Now that’s a privacy bill! The Hollings bill would, furthermore, create an Office of Online Privacy within the Federal Trade Commission and give federal protection to whistleblowers within companies that violate the law — protection that’s crucial, since frequently it takes insiders to reveal egregious privacy practices.

Which is why the gentlemen from HP and AOL were so opposed to it.

The hearing came hard on the heels of a report issued by the Federal Trade Commission that called upon Congress to pass legislation protecting online privacy — a report that identified notice, choice, access and security as the key elements of any policy. But in their prepared comments, Vradenburg and Cooper repeatedly said that they supported legislation that embodied only notice and choice. Cooper, speaking for HP, said that access was simply too hard to do — what’s worse, said Vradenburg, giving consumers access to their own personal information might make the information available to hackers as well. And security was too complex to mandate in a piece of legislation.

The give and take between the senators and the representatives of the computer industry was lively. Sen. John Rockefeller, D-W.Va., chastised the executives, saying that if brokerage firms can figure out how to let people trade stocks with security, then surely a company like HP can figure out a way to let people access their own personal information. And Sen. Richard Bryan, D-Nev., criticized AOL’s insistence that people who do not “opt-out” are giving their support to the company’s marketing practices. “That’s the effect of opt-out,” he said. “Silence is acquiescence. I don’t think that most Americans see that as an effective protection.”

Indeed, most AOL users don’t like those pop-up messages that try to sell you something every time you log in, but few AOL users take the initiative to navigate through the service’s screens to turn them off. Can you imagine somebody navigating to the AOL Marketing Preferences section and clicking the button, “Yes, I do want to receive special AOL member-only pop-up offers”? It’s like sending e-mail to a spammer: “Please send me your low-interest-rate credit-card offers.” That’s why companies like AOL are in favor of opt-out, rather than opt-in.

A few minutes before the session had to end, Sen. John Kerry, D-Mass., made an impassioned speech detailing a wide variety of threats to personal privacy. It was a good speech and I agreed with many of the threats that he identified — the dangers of having medical information, banking information and even genetic information flowing over the Internet, completely out of a person’s control.

But then Kerry said that the Consumer Internet Privacy Enhancement Act did the best job of protecting consumer’s privacy! And I just couldn’t believe what he was saying. Here was Kerry, the Democratic senator from the liberal state of Massachusetts, supporting a bill that did pitifully little to protect consumer privacy. I opened my jaw, stunned. But Kerry wasn’t finished: He stated that the continued growth of the Internet depends on information services remaining free, by which he really meant “supported by advertising.” Congress should be careful with any legislation that it passes, Kerry warned, lest it kill the free Internet.

This is the same argument that Doubleclick has been making through the Network Advertising Initiative: Don’t pass privacy legislation, or you will kill the free Internet. I raised my hand and said that Americans did not have to sacrifice their privacy in order to preserve the free flow of information on the Internet. “Just because the technology makes it possible to identify a person viewing a banner advertisement doesn’t make that an effective business model.” Instead of building comprehensive profiles, all advertisers need to do is advertise consumer electronics on electronics-oriented Web sites.

And that’s when McCain asked me his question. I stopped dead in my tracks, and stumbled through my first answer. Then I took a deep breath and tried again. The crucial issue had to be the selling aspect.

“In general,” I said, “I’m opposed to private companies selling information that is collected at taxpayer expense and that the government should be making available for free in electronic form.”

After all, there are many companies that are selling this information, from court cases to patents. It’s fairly big business in Washington, and if the government did a better job making the information available in electronic form, some of the arguments about advertiser-supported Web sites might fall away.

But McCain would hear none of it. He requested again that I answer his question, and when I couldn’t do that, he went on to the next speaker.

The hearings were over in another 10 minutes or so, but McCain’s questions nagged at me for the rest of the day. Clearly, the privacy of campaign contributors is violated when their names and that information is made publicly available. But once we have made the decision to make campaign contribution information public, the next question is “how will this information be used?” My answer, that this information should not be sold by businesses, but should be given freely in electronic form by the federal government, really had nothing to do with the privacy of the contributors.

As a privacy advocate, I inevitably feel some kind of disgust whenever lists of names and personal information are sold. But is it wrong? Perhaps not. If we want this information distributed, why not have private industry do it?

The real privacy issue, I realized, has less to do with the selling of the information, and more to do with what is done with the information after it is sold:

The Human Rights Commission had actually created two databases. The first was a detailed account of threats, thefts, beatings, mutilations, murders and massacres. This database was largely created from eyewitness testimony — more than 9,000 reports in all. The second was a database that tracked the careers of El Salvador’s police and military, built largely from official records, newspaper accounts and some personal recollections.

“What we were doing was tracking them by job, rank and unit from when they graduated the military academy as young lieutenants until they retired as senior colonels or generals,” recalls Ball. “And then we crossed these two databases, by unit and time.” The technique allowed the commission to develop “statistical human rights profiles” of individual officers and units. It showed how units became more violent when certain officers took control, and cataloged the crimes that had been committed under the watch of specific individuals. Essentially, the commission had created a Who’s Who of the nastiest criminals of the country’s 20-year civil war. “And then we published them in the newspaper!”

It was a bold move for a Yankee living so far south of the border. But the move was calculated. El Salvador was in the middle of a closely watched transition from military to civilian rule. “Because it was 1992, and not 1982, they didn’t blow up our office,” says Ball. Instead, the people who had been named in the files — most of whom by then were high-ranking officials — attacked the commission in the courts. And as for Ball, he left the country.

It certainly wasn’t what Ball had expected when he signed up to work as a peace activist in El Salvador after graduating from Columbia University. His first job in Central America was as a so-called nonviolent accompaniment. “You hang around with people who were likely targets of political violence, on the premise that your witness would prevent people who wanted to do political violence from doing it,” he remembers. “It’s interesting work, but it’s actually boring when you do it. They go to meetings, but you sit around out front” and talk to the secretaries.

It was these secretaries who gave Ball his first big break. To hear him tell it, the universal experience of secretaries in offices around the world is losing files on their computers. “If you can do anything to recover their files you become a computer expert.”

As it turns out, Ball is a computer expert. He paid for his undergraduate education by working part time as a database and statistics programmer. Soon after moving to El Salvador he took a job doing computer work for the human rights office of the Lutheran Church. From there he moved to the Human Rights Commission, where he designed the databases to track El Salvador’s bloody history.

At first glance, it seems odd that the Human Rights Commission would have massive data-processing needs. Perhaps it might need a few dozen paid researchers to interview the victims and then a small team of writers to assemble the findings in a big report — but is there really a need for SQL database programmers, forms designers and multivariable analysis? Sadly, the answer is yes.

In recent years the scale of atrocities in places like Guatemala, El Salvador, Rwanda, South Africa and Kosovo has been so massive that it defies comprehension by a single person. Each of these countries has seen hundreds of thousands of victims, jointly suffering millions of individual actions and crimes. While such overwhelming brutality can simply be written up — like the volumes of testimony and records cataloged by Argentina’s Commission on Disappeared People in the Nunca Mas report — such vast amounts of data cannot be easily made sense of in a Microsoft Word document. In El Salvador, researchers figured that if they could somehow capture these events, systematize them and put them in a data bank, they could produce summary reports showing trends, propose underlying theories and motives consistent with the data and ultimately draw a comprehensive portrait of the guilty.

“I remember when I lived in El Salvador in the early ’90s, I used to go to this Saturday afternoon drinking club at which there were these human rights lawyers, some journalists, and they would tell war stories to each other,” recalls Ball. “The collective knowledge among these guys was incredible — extremely detailed knowledge of who the intelligence service worked for and who was involved — but it wasn’t systematized. When we started putting it into databases, it became incredibly useful. It could be generalized to all kinds of purposes.”

Since then, Ball has worked around the world developing software that finds hidden patterns in large databases of people’s actions. He has worked for the Truth and Reconciliation Commission in South Africa, as well as in Ethiopia, Haiti, Guatemala and the former Yugoslavia. Ironically, he uses many of the same database-mining techniques used by marketing firms to manipulate consumer opinion or by intelligence agencies to track the movements of dissidents. But in Ball’s hands, these techniques instead become tools for justice and equity.

“I think that Patrick is doing very important work for human rights by essentially professionalizing human rights, by making it more of a social science,” says Fred Abrams, a senior researcher at Human Rights Watch. “Traditionally, human rights work has been through anecdotal case studies and narrative reporting based on field research. Patrick is addressing these issues in a more scientific manner. That’s crucial. It complements the narrative reporting.”

In 1996, Ball got a full-time job at the Science and Human Rights Program at the American Association for the Advancement of Science. At the time, the association’s main foray into the field of human rights was a project that used genetic fingerprinting to match up children who had been kidnapped during Argentina’s “Dirty War” with their grandparents. Another group of scientists was in El Salvador using genetic techniques to identify remains dug up from unmarked graves. Ball met some of the association’s scientists, who quickly realized that what he was doing fit their charter of applying science to the advancement of human rights.

“Technology has leveled the playing field between human rights organizations and intelligence services,” says Ball. “Back in the ’70s, intelligence services all over the world were getting pretty impressive computer hardware. This gave them the ability to track activities, peaceful civilian activists as well as violent [individuals], in pretty precise ways, to infer patterns and to use the data analysis as the basis for oppression.”

Today the same tools can be used to build an irrefutable record that documents a history of oppression.

Ball’s work is “incredibly important,” says Harvey Weinstein, associate director of the Human Rights Center at the University of California at Berkeley. “Patrick has the capacity with this statistical knowledge to develop hard, incontrovertible statistical data to provide the kind of evidence that people need to get a good sense of the kind of human rights violations that occur in these difficult situations. He is one of the leaders in the field of trying to develop and use statistics to provide substantiation for human rights abuses.”

Over the past decade, most of the large-scale human rights databases have been built either by Ball himself or by people he personally trained. Earlier this year, Ball co-edited a book called “Making the Case,” which discusses the technical decisions that were made, and the problems that were encountered, in building these databases in El Salvador, Haiti, South Africa and Guatemala.

“After I train people, I lose them to the private sector and to government,” says Ball. “By May ’99 those projects had all ended and pretty much wrapped up,” he says. “I wanted to preserve the technology memory of how these projects happened.”

More recently, Ball has applied his statistical techniques to analyze interviews with refugees from Kosovo. In 1999, in the midst of the NATO bombing campaign, several hundred thousand people fled their homes. Although the refugees said they were fleeing Serbian militias and Serbian government forces, many on the American left claimed that the refugees were actually fleeing NATO bombs. Ball, who analyzed data from border crossing surveys of 275,000 individuals, doesn’t believe this is true.

“The core finding was that there were three phases of exodus: March 24 to April 6, when there was a huge wave. Those people almost exclusively came from the south and west. Then it starts creeping up again and peaks on April 17, and those people are all coming from north central. Then it goes down to another low point on April 24; then it comes up again in late April and early May, and those people are coming from the south.

“My conclusion was that there is a pattern here, and that pattern does not match bombing at all. There has to be some centrally coordinating cause other than bombing causing migration” — presumably, armed paramilitary groups that were traveling from village to village.

“The findings were not a surprise to me,” says Abrams of Human Rights Watch. “The Yugoslav government was claiming that people were fleeing NATO bombs, which we knew was not the case because we interviewed hundreds and hundreds of people and that is what they said. But to have the numbers reach the same conclusion was very powerful and irrefutable.”

And that’s the reason Ball has been using the tools of data mining to bolster human rights causes for so many years. The statistical techniques turn individual accounts into hard data — and data can be used to argue a cause in a public forum or in a court of law.

“The notion ‘Never forget’ is an overriding principle” of his work, says Ball. “There has been a lot of psychiatric research that shows that individual victims have a much better outcome when the truth is acknowledged. The first level of goal is truth. The second level of goal is justice; if we know what happened, maybe there is some way that the perpetrators can be punished. The third is reconciliation. The fourth level is deterrence: ‘Never again.’”

Indeed, the statistical evidence can have a lasting impact on a nation that has been through the worst of times. In Croatia, Ball says, “some of the guys who held positions in the fascist government in the ’40s now hold positions in Parliament.”

But things are different in El Salvador, at least for some of the worst offenders from the country’s troubled past. Back in 1992, after Ball and his co-workers published their list of names in the newspaper and Ball had to leave the country, military officers whose names had appeared in print sued the Human Rights Commission for defamation. So the commission went to court with computer printouts of the 9,000 testimonies and presented them as depositions. It showed the court the statistics that it had used and its methodology, and asked to subpoena the army’s own records to confirm its allegations. “The officers withdrew,” says Ball. “They didn’t think that our methods were good enough, because they thought, ‘There is no way these guys can know these things.’” But the officers were wrong.

After the court hearing, the Human Rights Commission turned over its records to the so-called Ad Hoc Commission that was overseeing the country’s transition to civilian rule. One of the jobs of the Ad Hoc Commission was to come up with a list of people who would be barred from holding public office. The people on the Ad Hoc Commission’s list matched the list that had been supplied by the HRC.

Call it the triumph of the database.