013 Coke or Pepsi? Tricks with numbers, Part 2

Did you ever wonder about the warning label on diet soda: “Phenylketonurics: Contains phenylalanine”? Diet soda (along with many other food products) is sweetened with aspartame. People with phenylketonuria (PKU) cannot metabolize phenylalanine – the breakdown product of aspartame in the body. Phenylalanine accumulates in the brain, causing mental retardation.

PKU was discovered in 1934. Aspartame was discovered in 1965, and approved as a dietary sweetener in 1974. The FDA requires the warning label, because we already know about PKU (which is treated from infancy by going onto a low-phenylalanine diet).

But what would have happened if aspartame had been discovered first, and PKU was unknown?

PKU affects about 1 person in 10,000. Let’s assume that before the introduction of aspartame, everyone had an IQ of 100 (average), and the mean IQ for the population as a whole is also 100. After introducing aspartame, one infant’s IQ plunges to 25. The mean IQ for the population as a whole, including the 1 in 10,000 individuals with MR due to aspartame, would be: [(9,999x100) + (1 x 25)] / 10,000 = 99.9925. We would never detect such a small drop, just by looking at population-wide average IQs.

Mister Shady Broker, whom we discussed last time, generated a Type 1 error, “proving” a nonexistent relationship (his alleged ability to predict the stock market), by selectively omitting all the negative data (his wrong guesses). In the case of aspartame, we run the opposite risk: overlooking a relationship that actually exists (phenylalanine causes MR in some people), because the abnormal finding in one person has been swamped by large numbers of normal results. The risk of missing a relationship that really exists (in this case, phenylalanine causes MR in some people) is called a “Type 2 Error.”

Good research takes both Type 1 and Type 2 error into account during the design phase of the experiment, and when the results are reported, the authors will state the calculated risk of having made either type of error. (You can never eliminate the risks; you can just make them acceptably small.)

The debate about vaccines and ASD is shot through with both types of errors. The lay public, and practitioners of “junk science” tend to commit Type 1 errors; the safety data on immunizations are subject to possible Type 2 errors. More on this next time.