Let’s say you decide to toss a coin 10 times. The coin is evenly balanced, and for the first five flips you happen to get heads. So what’s the likelihood that the next toss will come up tails instead? The answer is 50/50, because, in any given toss, the coin is no more likely to turn up heads or tails. However, if you toss it long enough — 100 times, say, rather than 10 — the number of heads and tails you get will become closer and closer to equal.
Oh, probability! It’s true, as David J. Hand writes in his new book, “The Improbability Principle: Why Coincidences, Miracles and Rare Events Happen Every Day,” that it is “renowned” for its “counterintuitive nature, more that any other area of mathematics.” As Hand explains, humanity made “a major conceptual advance” when, in the mid-17th century, it began to understand “the step from the unpredictability of individual events to the predictability of aggregates of events.” Hand rates this insight as “on a par with the intellectual leap which led to the concept of gravity as a universal force acting between objects.”
But while probability remains difficult for many of us non-mathematicians to grasp, we have a much more pressing need to comprehend its basics than to master the principles of gravity. Chance and likelihood factor into every aspect of our lives from gambling on lotteries to the way we judge the risks of flying or buying stocks or taking a new prescription drug. When it seems like half the news articles we read include the phrase “studies show,” we really ought to have a decent sense of how such studies are conducted and what principles of probability they must take into account. Nevertheless, probability remains as hard for some of us to get a grip on as a wet and madly wriggling fish.
Hand, a mathematics professor at Imperial College London, offers such flailing readers a net with “The Improbability Principle,” and a lively and lucid one at that. The U.S. edition of the book is packaged rather like those pop-sociology bestsellers aimed at corporate marketeers (“The Tipping Point,” “Black Swan”), but it’s really a primer on probability with a special emphasis on rare events. Hand opens the book with an example: a story about the actor Anthony Hopkins, who in the 1970s got a part in a film based on a novel. He traveled to London to buy a copy of the book, but couldn’t find it in any of the stores he visited. Heading home, he discovered a copy of the very title he was looking for lying on a bench in the tube station. (Furthermore, Hopkins later discovered that this copy had once belonged to the author!)
Weird coincidences like this happen all the time, Hand explains, and for a complex tangle of reasons, even if some of us refuse to recognize them as chance occurrences. Maybe the Hopkins anecdote seems like little more than a freakish event, but what about the fact that both Abraham Lincoln and Caligula dreamed of being assassinated before they were in fact assassinated? What about the stories our friends and relatives insist on telling us about their premonitions and other “psychic” experiences, or the purportedly amazing predictions of psychics and astrologers? As Hand points out, before making important decisions, at least two modern presidents — Richard Nixon and Ronald Reagan — consulted people who claimed to possess such supernatural powers.
Most skeptics are familiar with the notion of confirmation bias — our tendency to take note of data that reinforces what we wish to believe and to ignore or forget data that doesn’t — and its role in fostering belief in various forms of prophecy and clairvoyance. That’s only one of the factors that contribute to Hand’s Improbability Principle, which is not a rule of its own, but rather “a consequence of a collection of more fundamental laws, which all tie together to lead inevitably and inexorably to the occurrence of such extraordinarily unlikely events.”
These rules include the law of inevitability (“even if each of the possible outcomes has a tiny probability of occurring, it’s certain that one of them will”), the law of large numbers (“with a large enough number of opportunities, any outrageous thing is likely to happen”), the law of the probability lever (“a slight change in circumstances can have a huge impact on probability”) and the law of near enough (“events that are sufficiently similar may be regarded as identical,” as in, the likelihood that a race will end in a tie depends on the accuracy of your stopwatch). There’s also the law of selection, a factor that Hand likens to shooting a bunch of arrows into the side of a barn and then painting targets around them afterward to demonstrate your extraordinary skill at archery.
A surprising number of these laws find examples and illustrations in the history of efforts to scientifically prove the existence of ESP and other paranormal abilities. Well, perhaps it’s not that surprising, given that most of the lab tests of telepathy have aspired only to show results slightly better than chance. (You have to wonder about the significance of a power that, even when it’s supposed to be working, doesn’t work all that often.) But Hand also reveals how essential it is to have a good grasp of probability’s role in far more mundane contexts.
Take, for example, a regional government’s decision to place traffic cameras at sites that have the worst accident rates, with the goal of intimidating drivers into slowing down and being more cautious. How meaningful is it if the accident rates at those spots decrease the following year? Only somewhat, because of a phenomenon called regression toward the mean. Some of the accidents at those sites were the result of conditions particular to those sites that made them more hazardous to careless drivers, but some of them just randomly occurred there. As a result, the next time accidents are counted, areas that had an unusually high number of them last time will statistically tend to have a lower number in the second count, even if the conditions at those sites don’t change. You need a statistician to help you figure out how much of that reduction is due to chance and how much to the effect of the traffic cameras.
“The Improbability Principle” is a book about math, obviously, a fact that is enough to entice some readers and to repel others. Its ideal reader will fall somewhere in the middle, neither too advanced to find it obvious nor too timorous to be put off by the occasional equation. Yet it’s an intensely useful (as well as a remarkably entertaining) book, and probably more so for those who tend to shy away from such topics as “not my thing.” It can transform the way you read the newspaper, that’s for sure. We all need to learn to think more deeply about chance, luck, randomness, the odds and all the other probability-related factors in our public and private lives, and if that isn’t always easy, there’s certainly no reason why it has to hurt.