Should national security depend on Michael Chertoff's gut?

On this topic, you write about drug couriers and the police officers who go after them.

Right, here you have a real expert who is able to pick a drug courier out of hundreds and hundreds of people floating through Los Angeles Airport. And when I asked him how he does it, he said, "I don't know -- I can't tell you because I really have no idea." The background is that he learned this skill while working over years with a very experienced colleague who told him, "Look at this person over there, he looks suspicious." Just by learning and watching he got this skill.

The problem, though, as you point out, is that those gut feelings aren't admissible in court. Juries don't trust them, and judges don't want police officers to say, "Well, I picked him out for a search because I had a gut feeling."

In the case of the American courts, the situation is something like this: If a policeman finds illegal weapons or illegal drugs, the policeman has to explain in court why he stopped the person in the first place. And the policemen that I talked to said they originally said, "Yes, we had a hunch there was something wrong with this person, and that's why we picked him out." When they said this in court, then usually the whole case was dismissed. The court expects reasons, and these experts cannot give reasons.

Of course, one wants to have protections of minorities and others, but in the end it's the wrong psychology that the court has.

But it's hard to say what the courts should do. Would you advocate that a court accept police officers' gut feelings? It's the same problem that we have with politicians -- we need to be able to judge this person's expertise before we can decide.

You're right, there is a conflict. I would suggest that in general the courts look at the accuracy and the experience of the policeman instead of whether the procedure was the right one.

The same thing applies to doctors. It would also be worthwhile to study how accurate a doctor's gut feelings are instead of dismissing them. And many American doctors feel that they have to hide their gut feelings from patients because otherwise they might be sued. Part of our research is about collecting evidence about the quality of gut feelings and showing when it works and what doesn't work.

One of the strongest bits of evidence pointing to the power of gut feelings is your study of high school dropout rates -- how a heuristic can be used to predict dropout rates.

This is just one study of about two dozen we have done and published in scientific journals. The exciting thing is that for decades people have argued that humans' intuitions follow just a few reasons and therefore they are irrational. So that was the received wisdom.

I've shown for the first time that intuitive judgment can actually be better than highly complex strategies. This was a big surprise, and the high school dropout study was one example of it.

The idea is that to make a prediction, you look at one very good cue, like in this case [the high school's] attendance rate. If two schools differ on the attendance rate -- one is high and one is low -- then you pick the high one, and you ignore all the other data. If they are equal on this you go to the second measurement, which was writing score. If they differ on that, then that's how you choose.

The real interesting result is that this simple strategy is, in the case of predicting which school will have a higher dropout rate, actually not just faster but also more accurate than a highly complex effort to take in all 18 measurements and try to weigh them optimally.

So you have all the data on all the schools in Chicago, and you want to predict which school will have the lowest dropout rate. You're saying it makes more sense to ignore some of that data, and look at just a few measurements to make your prediction.

That's the real fantastic part of this -- ignoring information can pay.

But why does that work? That seems completely crazy.

Well, that's the counterintuitive part about intuition. Why does it work? First I'll explain to you when it would not work. If the high school dropout rate were perfectly predictable from all the data and everything about it was understood, then the complex method of taking everything relevant and weighing it optimally, that would do better. On the other extreme it could be totally unpredictable, like a roulette game.

Most important problems are in between. So that means in order to make good decisions one needs to ignore part of the information and try to concentrate on the few cues that really are powerful predictors. Most of the things we know don't matter for the future. And the art is to concentrate on a few important things, then to have the courage to ignore the rest.

But isn't the hard part knowing how to rank them -- knowing which ones are the important ones to look at?

This reflects the kind of experience you need -- that's the expertise you need for doing predicting.

Now, in regard to Michael Chertoff, you've also studied 9/11, right?

[That's] a good example of where intuitions or gut feelings can go wrong, even cost your life. We know that after 9/11 many Americans stopped flying. What did they do instead? Did they stay home or did they drive? I analyzed the data and published a study showing that for 12 months after 9/11, miles driven was up 5 percent more than usual. And that cost the lives of about 1,500 Americans. They lost their lives while driving in an attempt to avoid the fear of flying.

These people were motivated by a kind of emotional gut feeling of fear that is well known. The fear is not of losing your life -- it's the consideration of where many people can die at one time. So that doesn't apply to driving. More than 40,000 people lose their lives in the U.S. on the road every year, but not many care much about it. But if it's a large number -- about 3,000 at one time -- that's really what causes fear.

The reason may be evolutionary. In the times when humans were in small bands wandering around, the loss of many lives at one point would have threatened the survival of the entire group. That kind of gut feeling is still around -- people are afraid of catastrophes, killer bees from west Africa and other kinds of things. Here's where an education of the public can help to avoid these fears the next time something happens.

Next page: How advertising exploits intuition