The other day while reading my local newspaper, I noticed an advertisement for the new Hollywood film Jumper. The advert proudly quotes a reviewer declaring, "This movie rocks!" Sounds terrific, right?
There was just one catch. Upon closer inspection, the quoted "reviewer" wasn't a usual film critic, but rather a novelist who happened to know Jumper's executive producer. The established critics judged the film rather differently. Andrew Pulver in the Guardian found it an "idiotic" film which "amounts to very little", while his colleague Philip French added that it is "ludicrous" and "incoherent". Legendary reviewer Roger Ebert of the Chicago Sun-Times scoffed that it is "so silly you may find yourself giggling helplessly", and Jeannette Catsoulis described it in the New York Times as "irritating" with such a "feeble plot" that "you'll be wishing that the pig had eaten the screenplay".
Somehow, none of those quotations made it into the newspaper advertisement.
To a professor of probability like me, that Jumper advert is a classic example of bias. The movie's producers, anxious to score at the box office, scoured the Earth for positive reviews while hiding all the negative ones. Since every single movie gets a positive word from somebody, somewhere, the appearance of praise in a movie advert tells us nothing -- absolutely nothing -- about a movie's true quality.
Bias affects us in many ways. For example, it is extraordinarily difficult to convince a friend to change their mind about any contemporary social or political issue. Why? Because they will invariably place great emphasis on any evidence which supports their position, while discounting those facts and events which contradict it. And this observation bias quite overwhelms whatever logical arguments can be mustered.
More worryingly, manufacturers of new medicines stand to gain billions if they can "prove" their drug is safe and effective. In some cases, they have commissioned multiple independent medical studies, and only published those which supported their case. This form of publication bias has become so serious that the International Committee of Medical Journal Editors declared in 2001 that they would no longer publish medical studies unless the independent researcher had complete freedom to reveal "the study's full results, including data perceived to be detrimental to the product."
People complain that they always find misplaced items "in the last place they look". Well, of course -- once they find it, they stop looking! This is a stopping bias -- we choose certain particular times to cease making observations, thus distorting our conclusions. In my basketball-playing youth, I scored 100% of the time on my final shot of the day. How? Simply because I stopped shooting after I finally scored! And an unscrupulous drug company could improve their results with a similar trick, by ending their medical study right after a string of lucky successes.
A somewhat overweight colleague of mine has noticed that when he attends an annual conference, many people ask him if he has shed some pounds (which he hasn't). It seems that due to the vagaries of memory, some folks think he's thinner and some think he's fatter, but the latter are too discrete to say anything. Call it the "politeness bias".
Sometimes biases are quite subtle. During World War II, the U.S. Air Force wanted to strategically reinforce the hull plating of its fighter planes to better withstand enemy fire -- but which parts of the plane should be reinforced? Charts and graphs were carefully constructed, showing the location of bullet holes on returning aircraft. The military then decided to consult a statistician -- always a clever move. Professor Abraham Wald immediately realised that those graphs were based on a biased sample: they only included data for the planes which actually returned from battle. The real issue was the location of bullet holes on the planes which were shot down and never made it home. The military wisely followed Wald's advice, to reinforce those parts of the hull that came back clean and bullet-free -- those were the places where any shots would be fatal.
A survey recently asked, "Which medium do you rely on most to keep abreast of the news?" An astonishing 62% of respondents cited the Internet -- far more than chose such alternatives as newspapers, radio, or television. This seemed to suggest a rapid and fundamental change in how society transmits information. But it didn't really -- the interview was conducted entirely on an Internet news web page! The respondents consisted exclusively of people who were already using the Internet to find news. It doesn't get any more biased than that.
Jeffrey S. Rosenthal is a professor in the Department of Statistics at the University of Toronto, and the author of Struck by Lightning: The Curious World of Probabilities.