Almost every time people start carrying folded umbrellas tucked under their arm, it rains. From this observation, a preschool child old might reasonably conclude that umbrellas cause rain. As grownups, we know better....or do we?
Picture this: One Monday morning, you receive the following unsolicited letter from The Shady Stock Brokerage Company. "Dear Future Millionaire: Here's your chance to live the life of your
dreams. I have a foolproof method of predicting the stock market, and I have selected you as a potential investor. But I wouldn't accept your money today, even if you begged me. First, I want to prove to you that my system works. I predict that on Friday of this week, the stock market will close down from what it is today." Sure enough, on Friday the market closes down. The following Monday you get another letter: "This week, the market will close up." Sure enough; it closes up. The third week the guy predicts "up" again, and sure enough, the market closes up. The fourth week he predicts down, and it's down. The fifth letter says "Now do you believe me? I've predicted the market correctly four weeks in a row. Now let's talk business. Send me $1000 and I'll make it grow for you, same as I've done for all my clients, and we'll all get rich together. If I don't hear from you within 48 hours, I will assign your slot to someone else, and you will never hear from me again." You hesitate for a moment, then send off a check. After all, he's been right four weeks in a row, and you have the letters to prove it.
You've just been suckered by one of the oldest scams in the book.
What you don't know is that each month, Mister Shady Broker sends out 800 letters to a new batch of potential victims: Half of the letters say that the market will close up; the other half say that it will close down. At the end of the week (after reading the Friday newspaper to see what the market did) he crosses 400 names off his mailing list - the people who got the letter containing the incorrect guess. Week 2, he repeats the process, with the remaining 400 names. Mister Shady has no idea what the market will do, but he can count on the fact that 200 letters will be correct; at the end of the week, he crosses the other 200 names off his list. Week 3 he sends out 200 letters, and at the end of the week, he crosses off another 100 names. Week 4, he sends 100 letters, and crosses off 50. The 50 people who remain on his list have now received four correct predictions in a row; these are the people to whom he makes his pitch for money.
From the vantage point of those 50 people, the truth seems incontestable. They have the four letters as proof! Of course, they don't know that the other 750 people even exist - that's what makes the scam work in the first place. As far as they are concerned, this guy has predicted the market with infallible accuracy.
Mister Shady's crowning touch is to invite all 50 people on that final list to a free dinner and investment seminar. He includes a few people who got 3 out of 4 correct predictions, to give the appearance of "balance." (After all, if everything were letter-perfect, people might start to get suspicious.) Mister Shady might even invite some potential victims who haven't received any letters at all, carefully seating them next to people he has already hoodwinked. As the enthusiastic recipients of Mr. Shady's letters chat up his skills to one another, the newcomers catch the fever and are drawn in as well.
A huge proportion of the fear over vaccines, and a huge proportion of the hype over various unproven therapies, bear a strong resemblance to Mr. Shady's stock market scam: Practitioners of sloppy science create the appearance of cause-and-effect, when no proof really exists. Victims are sincerely convinced that a cause-and-effect relationship exists, and cite their personal experience as proof. The victims band together and share their stories with one another, reinforcing each another's beliefs. And they draw in newcomers with the intensity of their beliefs. The same logical fallacies and emotional dynamics are at work as in Mr. Shady's get-rich-quick scheme.
Technically speaking, Mr. Shady's ploy draws the victims into making a "Type 1" error: concluding that a relationship exists, when it really doesn't - In this case, erroneously concluding that Mr. Shady can really predict the future, when in fact, his guesses were totally random.
But life isn't quite that simple.
There is another side to the coin, technically know as a Type 2 error. Failure to account for Type 2 error leaves a hole in the scientific data big enough to drive a truck through. More on that in our next post.
