Pity the errant expert. Neurosurgeons, nuclear physicists and fighter pilots are at the top of their field, but they fall victim to the same errors in judgment and reasoning that we all make. And then some. A brain surgeon assumes she's found a glioblastoma because she's seen it so many times; a pilot homes in on a landmark that he thinks he remembers from a well-traveled flight path. Gordon Rugg, a 49-year-old psychologist who teaches computer science at the University of Keele in England, argues that science's thorniest problems, from the causes of Alzheimer's disease to the presence of extraterrestrial life, may be solved by systematically examining kinks in experts' intellectual armor. To that end, Rugg has devised Verifier, a theory that calls for top-down review of how experts tackle problems. Here's how it works: First, you search for knowledge gaps and basic errors in reasoning; such gaps are common, because specialists are often ignorant of a critical, related domain. Then you figure out how experts acquired their information and just what they have deduced from it. Formal logic is applied again, this time to determine whether a premature or erroneous idea prevails. Verifier was road tested in 2004 on a dazzlingly difficult problem: the Voynich manuscript. Dubbed the "most mysterious manuscript in the world," the 400-year-old document long confounded cryptographers and historians with its indecipherable -- but apparently rule-based -- language. Rugg studied the Voynich and the multiple theories it has spawned, and promptly concluded that the manuscript is most likely an ingenious hoax.
KP: People have argued for hundreds of years that the voynich could be a hoax, but you were the first to actually demonstrate this. How did you turn a corner that everyone else crashed into?
GR: The language of the manuscript, known as Voynichese, clearly behaves in non-random ways, and nobody could think of a plausible mechanism to generate a hoax. It looked as though a hoax would have to be as complex as the code, if there were a code. I started wondering whether the non-random text was a result of the technology used to make the manuscript. The only way to check that was to examine the tools and techniques that existed in the 15th or 16th century, when the manuscript was likely created.
why didn't anyone else realize that Voynichese could be a meaningless by-product of the device used to make the manuscript?
Several interesting psychological things were going on. One was confirmation bias. Everyone had their pet theory and they looked for confirming evidence to support it. Secondly, people were using conceptual tools of our period like probability theory. No one living in the 15th or 16th century would think in terms of probability theory for combining characters or syllables. So that got me thinking about "affordances," the idea that a technology may have side effects that are completely unintended by the designer of that technology. I looked for -- and found -- a device that could have generated regular syntactical patterns of the sort found in Voynichese. Also, no experts on hoaxing had examined the manuscript. It's a classic fact that if you're outside your own area of expertise, you don't perform better than a novice. So an expert cryptographer is not a judge of what you can and cannot hoax.
you teach computer science, yet you used pen and ink to unlock the manuscript. why not design a program?
There are all sorts of things you can do easily with paper-based technology, things for which you would have to program software: Manually you can generate text moving left to right across the table, or right to left, equally easily. This is where other people examining the Voynich manuscript missed possibilities. Technology channels us down a particular way of thinking.
Experts often rely on pattern matching: "this looks like the flu; that sounds like a viral infection." But you state that this can get them into trouble. How do you avoid these mistakes, given that pattern matching is a more instinctual mode of thinking than is sequential reasoning? The question is, when is pattern matching leading us astray by being a bad master rather than a good servant? In surveying the manuscript, I used pattern matching myself: "This looks like a by-product of technology; that looks like a confirmation-bias problem."
where else have mistaken assumptions derailed experts?
Archaeologists learn about prehistoric farming practices by calculating the age at which animals were slaughtered. Researchers use a set of standard tables that originated with a 19th-century German archaeologist, who got it almost right, but not quite. For a century everyone has been replicating his errors and systematically misjudging the age of certain species as a result. Similarly, in formulating the equation of the neutrino, physicist Richard Feynman discovered that everyone was quoting from one paper that was itself incorrect. Said Feynman: "I never pay attention to anything by experts.' I calculate everything myself."
how does verifier differ from other research into expertise and error?
Verifier is novel because it integrates disciplines such as expert reasoning, human error and logic. But there's good work out there: A British team recently analyzed models of pilots' manuals and found that there were system configurations that were possible but weren't covered, as well as configurations that the manual described ambiguously. So it seems very unlikely that pilots would be able to determine in-flight how to handle a plane that got into those configurations.
Tags:
behavior,
brain surgeon,
cryptographers,
experts,
fighter pilots,
flight path,
formal logic,
historians,
hundreds of years,
kinks,
logic,
reasoning,
university of keele,
work
