Spain won this month's football World Cup, but Paul the Octopus won a second, hotly contested title that paralleled the main event. Paul correctly predicted the result of Germany's seven matches at the World Cup, and then predicted that Spain would beat the Netherlands in the final. Before each match, Paul's handlers placed two boxes of food in his tank, each box depicting the flag of one of the two teams playing in the upcoming match. Paul ate the food (a mussel) from the box that depicted the winning team's flag on all eight occasions, earning him the title of oracle and a significant Wikipedia presence. Paul's "colleagues" at the Chemnitz Zoo were failed prophets: Leon the porcupine picked Australia to beat Germany (Australia lost 4-0); Petty the pygmy hippo picked Germany to beat Serbia (Serbia upset the Germans 1-0); and Anton the tamarin ate a raisin representing Ghana (who lost to Germany 1-0). Paul's legacy bestowed mixed fortunes on octopushood: on the one hand, Parry Gripp wrote a YouTube smash hit called "Paul the Octopus;" on the other hand, octopus consumption has skyrocketed in China.
People are fascinated with oracles, with the ability of man and beast to divine the undivinable relying on a combination of intuition and
obscure rule-based algorithms. Life is a series of confounding coincidences and inexplicable flukes, so we're drawn to anything--oracles, prophets, rulebooks, statutes, religious
doctrines--that purports to make order of the chaos.
Our preference for patterns infects our ability to think clearly in numerous situations and contexts. Night after night, midway through World War II, London was bombarded by a series of air raids led by German V-1 bombers. The bombs created havoc, and bedeviled Londoners dealt with the chaos by trying to divine which areas of the city had been targeted and which would be targeted on future raids. Experts plotted each strike on a map of London, and began to perceive patterns in the data. If you cut the map into quadrants, many of the bombs seemed to be falling near the River Thames; perhaps the Germans had planned to break the banks of the Thames. In truth, no matter how badly Londoners wanted to believe that the Germans weren't interested in bombing their area of the city, the bombs fell randomly. There was no strategy, and any patterns were coincidental and certainly didn't reflect specific intent.
The same bias for patterns colors how we view sporting prowess, and even the outcome of games of chance. In a now classic paper, Tom Gilovich, Robert Vallone, and Amos Tversky examined the claim that basketball players sometimes become "hot," temporarily sinking a string of baskets that far outstrips their usual level of accuracy. Basketball fans are convinced that the hot hand effect exists, but Gilovich and his colleagues showed that the hot hand was a fallacy, an illusion that arises because we tend to see patterns where they don't exist. The hot hand was missing from the shooting record of one NBA team, the Philadelphia 76ers, from the free throw shooting record of another, the Boston Celtics, and from the men's and women's Cornell varsity teams. Researchers continue to question whether the hot hand exists in basketball, tennis, golf, and other sporting domains, but a newer finding in one domain makes it's impossible to deny that people perceive illusory patterns: games of chance in gambling. In 2005, I visited Atlantic City with Danny Oppenheimer, a cognitive psychologist at Princeton. We interviewed gamblers as they left one of the large casinos on the boardwalk, and, almost without fail, they recounted times when they experienced a streak of luck. We focused on the game of craps, in which players win when they roll certain numbers with two dice. The beauty of craps is that it's entirely based on chance: any "streaks" must be coincidental. Some respondents claimed that luck emerged from the ether, without warning; others claimed they had won more often after blowing on the dice using a finely honed technique; and, still, others claimed that a particular friend had a way with the dice, generating so many wins that his "skill" suspended the laws of probability.
Why do these illusions persist in the face of repeated disconfirmation? One reason is that they're incredibly vivid and compelling. If you ask basketball fans to list the ten games throughout history that loom largest, they're likely to point to games that featured impossible comebacks, vanquished foes, record demolitions, and freakish individual performances. In the end, ten great games probably soak up 90% of that part of their brains devoted to "past basketball games," while thousands of nondescript games drop off the radar completely. A second reason, also documented by Tom Gilovich, is that we have a strangely skewed idea of what randomness means. Suppose you toss a coin twenty times and record the outcome after each toss. Which of the following two series seems most representative of what you'd expect to find?
Most people think a sequence with the number of outcome shifts (changes from X to O or vice-versa) that matches the second streak is more representative of a randomly-generated string, largely because it doesn't have any long streaks of Xs and Os. In truth, though, a sequence with fewer outcome shifts (approximately 10 shifts in a series of 20 coin tosses) is more representative. You'd expect there to be some longer strings of Xs or Os, because the outcome of the coin toss should only change about 50% of the time (since there are two possible outcomes). In fact, people intuitively believe that randomness is best captured by a process in which the outcomes change 70% of the time (more like the second string). In other words, our concept of randomness features too much alternation; as soon as a basketball player scores three times in a row, a craps player rolls three winning outcomes in a row, or three bombs fall in a cluster, we're convinced that the outcomes are following a pattern.
There's a silver lining, though. Some of the patterns we see may be illusory, but others lead to some of the greatest discoveries we've known. When the rest of London attributed the cholera epidemic of 1854 to "miasma," or bad air, physician John Snow recognized that the cases conformed to a pattern: they happened to cluster around a public water pump. Germ theory didn't emerge for another seven years, but Snow's insight led to the closure of the pump, and may have curtailed the spread of the outbreak. Like Snow, Archimedes had access to the same information available to great minds that preceded him, but only Archimedes noticed that the water in his bathtub systematically rose and fell as he entered and exited the tub. Thus was born a new method for determining the volume of objects of irregular shape.
The London Bombing example comes from a book by Tom Gilovich (1991). How We Know What Isn't So: The Fallibility of Human Reason in Everyday Life.
Review of the Hot Hand Fallacy: http://pages.stern.nyu.edu/~aalter/sports.pdf