- Research findings suggesting that TV watching may cause attentional problems in children have influenced the public conversation.
- There's an ongoing debate in the scientific literature about the strength and nature of the link between TV and attentional problems in children.
- New research suggests that TV likely does not cause attention problems in children.
- Science produces conflicting and at times erroneous results. This is not a reason to abandon science, but to improve it.
Because of its plodding, zigzagging, fussy, and skeptical nature, science often frustrates us, appearing unable to make up its mind. Scientific recommendations often conflict, reverse themselves, or come with myriad qualifications and stipulations. The whole process can be daunting. A bit like the stock market, science includes dark days and course corrections, and is susceptible to mania and manipulation.
Part of the problem lies with the nature of scientific inquiry itself. It’s hard to do good science. There are many ways to fail — unrepresentative samples, leaky designs, imprecise measurements, and careless interpretations, to name but a few.
While some pitfalls are internal to the way science is done, others relate to the way science is communicated. Because complexity and equivocation are less palatable to our attention than simplicity and brash conviction, scientific findings are often packaged for popular consumption as catchy, headline-grabbing slogans that fail to capture the nuances of the actual results.
Complicating matters further is the fact that publicized findings, once out there, take on a life of their own, often becoming received wisdom that is no longer questioned and therefore difficult to dispute. As in politics, an initial declaration is more potent than the eventual retraction. Once we’ve come to believe something, making us change our minds is difficult. This is almost always true, and even more so in cases when the science happens to confirm what we intuitively suspect or fear.
One case in point is the notion that early childhood television exposure causes later attention problems. The link was first made and popularized in a 2004 study by Dimitri Christakis and colleagues at the University of Washington, Seattle.
The authors used data from the National Longitudinal Survey of Youth (involving 1278 children at age 1 and 1345 children at age 3) to look for potential links between the amount of TV children watched at ages 1 and 3 and attention problems at age 7. Measures of both TV time and child inattention were based on maternal reports.
Results showed that “hours of television viewed per day at both ages 1 and 3 was associated with attentional problems at age 7.” The authors concluded: “Early television exposure is associated with attentional problems at age 7. Efforts to limit television viewing in early childhood may be warranted.”
The original paper is quite careful in acknowledging the limitations of the study. The authors noted that additional research was needed. They acknowledged some of the study’s limitations, including the reliance on maternal report, which may be inaccurate, and the absence of data on the content of TV children had watched, which may, potentially, account for TV effects more than time per se.
They also noted: "We cannot draw causal inferences from these associations. It could be that attentional problems lead to television viewing rather than vice versa… It is also possible that there are characteristics associated with parents who allow their children to watch excessive amounts of television that accounts for the relationship between television viewing and attentional problems.” (For example, less-competent parents may both allow more TV watching and create a home atmosphere that promotes child dysfunction.)
However, subsequent reporting, as will often occur, ran with the attention-grabbing message: Early TV caused ADHD. As Matthew McBee of East Tennessee State University and colleagues note: “Using Google search in April 2020 for ‘Does TV cause attention problems,’ most of the top hits, including some from reputable sites such as WebMD and whyy.org, claim a link between TV and attention problems. WebMD uses blatantly causal language in its headline (‘Toddler TV Time Can Cause Attention Problems’).”
Given this, McBee and his colleagues recently (2021) decided to take a fresh look at the original data set by using a “multiverse analysis” approach, which amounts to applying multiple analytic strategies to the same data set, in search of converging evidence for a given conclusion. The idea is that since any single analytic strategy necessarily includes weaknesses and limitations, subjecting the data to multiple different analyses may help ascertain the robustness of the conclusions.
In the authors' words: “A significant finding presented in isolation can be misleading if it is not representative of larger set of possible results that could have been obtained under alternative but equally principled analytic approaches. One way to evaluate the dependence of a claim on a specific model is to subject the data to a wide variety of defensible analyses, systematically exploring how sensitive the outcome is to different model specifications.”
By way of analogy: A movie date may go well, yet concluding that the couple is compatible may be premature. A movie date contains certain parameters (darkness; sitting side by side; proximity to other people; popcorn etc.) that uniquely shape the young lovers’ interaction. Yet if the same couple go on multiple different dates (nature hike, restaurant, car trip, dancing, escape room, etc.) and come out satisfied in most of them, then we can more safely assume compatibility.
Deploying this multiverse approach, the authors ran the original data set through no less than 848 distinct analyses. They reason that if TV time is indeed linked to hyperactivity, then most of the analyses will produce significant results in the predicted direction, and the more rigorous analyses will produce stronger such results.
Actual results, however, showed that of 848 analyses, only 166 (19.6%) produced evidence of a relationship between the variables. Further, only 21 (4.8%) of 440 analyses deemed best by the researchers produced significant results. The authors therefore assert that the original study's claim "is not robust and is unlikely to be true." They conclude: "TV likely does not cause attention problems.”
This of course is not the last word. The last word is rarely spoken in the social sciences, in part because our measures are still fairly crude, and in part because social facts and processes are dynamic and context dependent: They change with the changes in society.
Yet the fact remains that when it comes to refereeing between competing truth claims, science, for all its pitfalls and frustrations, is by far our best option. To paraphrase Winston Churchill, science is the worst method of discovering truth, except for all the others.
By way of analogy: The fact that several of your love relationships ended in heartbreak and confusion is not a reason to renounce love. It is a reason to learn to love more and better. Likewise, the fact that scientific inquiry produces conflicting, confusing, and at times erroneous results is not a reason to abandon science, but to strengthen it and recommit to doing it better.