National Research Council on data mining for terrorists

"No invasion of privacy is justified if the program doesn't work."

Published October 8, 2008 10:00PM (EDT)

For the past several years, millions of our tax dollars have been spent on analyzing the digital behavior of potential terrorists in order to keep us safer. In other words, just as your supermarket can easily track which toothpaste you buy through the use of loyalty cards, so too can the feds use similar data based on places you've traveled, phone calls you've made, e-mails you've sent, money you've spent and so forth to determine whether you're a terrorist. Since then, we've seen example after example after example that the terrorist watch list is broken.

Tuesday, a 352-page document outlining the results from the National Research Council determined that while "these methods have been useful in the private sector for spotting consumer fraud, they are less helpful for counterterrorism precisely because so little is known about what patterns indicate terrorist activity; as a result, they are likely to generate huge numbers of false leads. Such techniques might, however, have some value as secondary components of a counterterrorism system to assist human analysts. Actions such as arrest, search, or denial of rights should never be taken solely on the basis of an automated data-mining result."

So my question is a pretty basic one: Didn't we know this already?

I mean, let's review the facts:

This summer, the ACLU announced that the terrorist watch list had hit 1 million names. (The Transportation Security Administration says there are only 400,000 actual people on this list.) Without knowing anything scientific about profiling of terrorists, I can't imagine that there are 400,000, much less 1 million, bona fide terrorists out there, just waiting to get us. That's one out of 750 Americans -- 300 million divided by 400,000 -- assuming that none of them are foreigners, a point I'll get to in a moment. (Thanks for checking my math, guys!) C'mon guys -- we were attacked by 19 hijackers who took out 3,000-plus people in New York and D.C. -- it's not like these guys need three times the number of soldiers that we have in Iraq to come after us.

As Timothy D. Sparapani, an ACLU senior legislative counsel, told me in an e-mail earlier today:

The NRC report validates the ACLU's longstanding claim that data mining for anti-terror and law enforcement work is worse than junk science, it is pseudo-science and in the same league as alchemy and astrology. Government agencies that are spending billions on data mining to fight terror would spend their money more effectively trying to discover a magical formula to turn lead into gold.

Professor Fred Cate, one of the lead authors of the paper, agrees.

I asked him by e-mail if the conclusion of this study was something that we, the public, knew already on a gut level -- and he concurred.

In fact, the report acknowledges that most of its recommendations originated with other groups who have addressed these issues," he writes. "What the report tries to do is build a broader case for them -- especially noting that implementing them will help not only privacy but also security (after all, what good is it to stop the wrong people?) -- and lend the authority of the National Academy of Sciences to the call for Congress and the Administration to act.

Furthermore, the Feds tell us not to worry, that most of these folks are foreigners away, and yet, the intelligence bureaucracy seems unable to identify actual foreign terrorists or Americans who share the same name. This is why U.S. attorney James Robinson is continually profiled as a potential terrorist.

As the ACLU puts it:

In fact, the listing of a terrorist named "Jim Robinson" may not be a "quality-assurance" problem at all; it may be that there is a genuine terrorist with that name and that the evidence against him is very strong. But Jim Robinson the former head of the Criminal Division of the Justice Department continues to be affected by this list. The fact is, the government often simply does not have a mechanism by which to distinguish individuals with the same name (and experts say it is surprising how often two people with the same name will also possess other information in common, such as date of birth).

That's of course aside from the fact that there are loads of ways, as security expert Bruce Schneier points out, to defeat American security measures -- some of which are as basic as changing one's name. Oddly, this appears to totally defeat the American government watch list. Not to mention the fact that there are plenty of ordinary consumer activities that really don't match up to terrorist activity at all.

And in the end, that's what matters -- this data-mining stuff just doesn't work, says Cate. As he wrote to me in an e-mail today:

For me, as a lawyer and law professor, the biggest surprise was that the technical and scientific members of the committee (21 members in all, only three lawyers) expressed such strong concerns about the scientific validity of many data-based programs and whether they work in practice. Prior research has suggested that there may be privacy issues raised by those programs, but I didn't expect to hear such skepticism about whether they actually work. From my point of view, a program that relies on personal data but then fails to detect or prevent terrorism necessarily constitutes an unwarranted invasion of privacy. In short, no invasion of privacy is justified if the program doesn't work.

By Cyrus Farivar

MORE FROM Cyrus Farivar

Related Topics ------------------------------------------