Verified by 
Millions of Americans were glued to their TV sets on the morning of September 11, 2001 and on the following days, watching live coverage and repetitive replays of scenes that they would never have expected to see outside of a disaster movie. High-profile claims were subsequently made that American had suffered prolonged negative mental and physical effects of these events, exacerbated by excessive viewing of them on television.
Further claims were made that the mental and physical damage did not depend on television viewers being at immediate direct risk from the events at the World Trade Center in New York and Pentagon in Washington DC or having loved ones being in danger or actually being killed or hurt. A case was being made for a virtual trauma conveyed by the media and that the nationwide effects of 9/11 were proof that posttraumatic stress reactions could be elicited and persist in the absence of a direct exposure to events. If valid, these claims would suggest the need to expand the boundaries of events considered necessary to cause posttraumatic stress disorder. The prevailing wisdom had been that it required direct exposure or direct personal relevance of an event, such as the violent death of a spouse or child, to cause posttraumatic stress reactions. Otherwise, any psychiatric disturbance would represent something else, maybe a general anxiety disorder or major depression.
This is a crucial distinction. The symptoms of posttraumatic stress response are highly subjective and have considerable overlap with general distress, including normal responses to everyday stresses. Similarly, the symptoms of posttraumatic stress disorder overlap considerably with other psychiatric disorders. Presumably, what is distinctive about posttraumatic is a demonstrable tie of the symptoms to a serious event.
One set of claims appeared in the high-impact JAMA: Journal of the American Medical Association. Another claim that the events of 9/11 and their viewing on TV caused heart ailments appeared in a later companion piece in the Archives of General Psychiatry. The American Psychological Association was sufficiently impressed by these claims to fly one of the authors of the two articles, Roxanne Silver to Washington DC to testify before Congress and then invited her to contribute an unprecedented three papers to a special issue of the American Psychologist devoted to the psychological effects of terrorism.
With the 10th anniversary of September 11, 2001 upon us, it would be good to take another look at the evidence that were mustered for these claims, how valid the evidence was at the time, and how this claim stood the test of time. This post will be the first in a series examining the evidence presentedfor these claims in the two articles, how well these claims fit with the rest of the scientific literature, and the implications for just how skeptical a view we should adopt for what we read in the scientific literature and how it is portrayed in the media.
I am going to make case that the authors of these two articles did not present strong evidence that the American people suffered significant mental and physical negative effects from simply learning of the events of September 11, 2001 or watching them on television. These horrific events profoundly changed Americans' lives, but learning of the events or watching on television did not have public health or clinically significant mental and physical effects, if people were not directly affected or in the immediate vicinity of the events. I am going to make a further argument that the authors of these two articles made it quite difficult to compare their claims to other evidence, but it is not impossible, and that other, better quality evidence can be shown to contradict their claims.
The articles are freely available, and so if you would like to do your own skeptical sleuthing and compare your conclusions to mine, you can download them and read them before proceeding. I am going to start with the JAMA article, and I will discuss the Archives of General Psychiatry article in later blog posts. Just click on the highlighted links in this blog post to obtain the articles.
The JAMA article claims that 17% of the US population living outside of New York City had posttraumatic stress symptoms related to September 11 two months after the events and that 5.8% of the population still had the symptoms at six months.
Let's take a look at the evidence that is presented in the article. We will see that there are substantial problems with the measures and with the sample of Americans living outside New York City on which these claims are based.
The study built upon an ongoing project in which participants received free Internet access in return for filling out web-based surveys three or four times monthly. The authors took advantage of this arrangement to administer questionnaires to capture participants' reactions immediately after September 11 and in the following months and years.
Measurement Issues. The measure of Acute Stress Reaction used in this study was a modification of a questionnaire used in other studies, the Stanford Acute Stress Reaction Questionnaire (SASRQ). You can examine a copy of the original questionnaire by clicking on this link. The authors claimed that items were scored in terms of the American Psychiatric Association Diagnostic and Statistical Manual Of Mental Disorders, Fourth Edition (DSM-IV) criteria for Acute Stress Disorder, but that is not quite accurate.
The DSM-IV requires that symptoms of Acute Stress Disorder have a duration of 2 days to 4 weeks and interfere with functioning. The reason for the duration criterion is to exclude either transient symptoms or more enduring symptoms that may indicate a chronic disorder, not an acute one. In addition to inquiring about the duration of symptoms, the original questionnaire allows respondents to indicate how frequently they experienced the symptom in a specified time period using a six-point scale ranging from "not at all experienced" to "very often experienced". However for the purpose of the 9/11 study, the authors did not ask about duration and they provided only a simple yes/no option for whether respondents had experienced particular symptoms. In doing so, the authors lost considerable crucial information.
It is not clear why the authors did what they did to the existing measure, but we can safely assume that respondents from the normal general population will often be reporting on symptoms only lasting a day or chronic recurring symptoms independent of an event, particularly when they are asked to evaluate an event that did not directly affect them personally.
A diagnosis of Acute Stress Response requires that people meet an impairment criterion-- that their personal functioning was negatively affected by their symptoms. This is intended to exclude reactions that can be considered part of a normal reaction. Unfortunately, the authors of the study did not collect information as to whether the complaints respondents were reporting actually interfered with their functioning
The items on the questionnaire cover quite general complaints. Look at the questionnaire and you will see that items assess such complaint as feeling restless, difficulty falling asleep, and not wanting to talk about the event. Interpretation of these phenomena in terms of a stress response assumes that someone - in the case of a questionnaire, the respondents -- can accurately distinguish whether the symptoms only came after the event and can be attributed to the exposure to particular event. Yet, note that for the purposes of the study the authors are suspending the assumption that direct exposure is necessary. Before participants were confronted with the web-based questionnaire, they may never have even thought to attribute their symptoms to the events of 9/11, and they may simply have been noting that they had symptoms when they completed the questionnaire, regardless of their presumed cause.
The advantage of relying on a standardized questionnaire is that readers can make independent comparisons to data collected from other populations. In this instance, it would have been informative to know how other people in other situations score. However, the authors' modification of the questionnaire precluded either making comparisons to diagnostic criteria or two other populations. So, for this questionnaire at least, the authors are asking that we rely on their interpretation without being able to check it directly against evidence from other sources.
In order to assess longer-term reactions to September 11, 2001, the investigators used the Impact of Events Scale-Revised (IES-R). It is a widely used and widely misinterpreted in terms of "posttraumatic stress symptoms." It consists of 22 items assessing intrusive thoughts, avoidance, and arousal. Like the Acute Stress Reaction Questionnaire, the items are quite nonspecific and often can reflect variations in normal functioning. For instance, cancer patients score high (but not as high as one would expect) not because they are necessarily ruminating about the past and having intrusive thoughts, but because they are vigilant about new signs and symptoms, as well as attuned to information that they are receiving from their oncologist. So, the measure picks up an appropriate vigilance in cancer patients, not abnormal intrusive thoughts and ruminations, as it was intended.
A provocative study by psychologist Paul Lees-Haley revealed a further limitation of the IES-R. He asked college students to think of the worst TV program they had seen in recent weeks and then to complete the IES-R with the TV program as the event. It did not have to be a horror movie. In fact, the students sometimes picked bad situation comedies that grated on them. Yet their IES-R scores were higher considerably than what was obtained in the Silver et al study and in the range often seen in populations with true exposures to traumatic events. This seems to point to the ambiguity of a high score: it can be obtained in the context of a passing reaction to a normal event, and so we need to know for sure that reactions to a seriously threatening event is being assessed, in addition to this event being surely in the past.
The authors of the 9/11 study apparently collected data with the original IES-R instrument, but then modified their scoring of it so that it is difficult to make precise comparisons with other research. Their rescoring made an endorsement of "moderate" of items equivalent to an endorsement of "extreme". But the overall scores they obtained from the 9/11 respondents still do not seem particularly high. Fortunately these authors also measured other psychological phenomena with other questionnaires, and we can use these data to examine claims about their results obtained with the modified measures.
One was a measure of general distress, the Hopkins Symptom Checklist, for which data from other populations is available. The scores obtained in the 9/11 study were substantially lower than established clinical cut-points indicating the possibility of disorder, and they were lower than scores obtained with general medical populations. For instance, when the Hopkins Symptom Checklist is administered to patients in a family physician's waiting room, about a third of them score above the standardized cut point for distress, considerably higher than participants in the Silver and colleagues' study. So, results with the Hopkins failed to corroborate the authors' claims that they were assessing severity of symptoms representing disturbance outside of the normal range of distress.
The Brief COPE, a measure of coping was included with the intention of understanding whether what respondents did to cope with the events of September 11 could reduce or exacerbate their stress response. You can examine a copy of the instrument by clicking on this link.
You can see that the items are rather generic and respondents might have had difficulty completing this instrument with respect to the events of September 11. Scores on the self-blame items ("I been criticizing myself." and "I been blaming myself for things that happened.") had among the highest associations with distress following September 11. What could this strange finding mean? Why would some individual Americans possibly be blaming themselves for the events of September 11? One possible explanation of the association is that the coping measure was inadvertently tapping into neuroticism. Characteristically, neurotic persons blame themselves and they may have been reflexively doing this in completing the coping questionnaire, ignoring that responses specific to September 11 were being sought. Consistent with that hypothesis, the association between stress response and how much television coverage of 9/11 was watched disappears when coping is taken into account statistically. Maybe it is simply a matter of neurotic persons being prone to distress, coping less adequately than other persons, and watching more television coverage of 9/11.
Selection Bias: An Unusual Sample? Just who were these participants in the web-based survey? The authors claim that was a national probability sample so that that generalizations could be made to the rest of the American population and their abstract claimed a petition rate of 78%. However, a careful reading of their method section suggest that they were only able to retain a small minority of the persons originally sought for the web-based study. Apparently, the incentive of free Internet access or coupons was not sufficient to maintain respondents in the continuing survey. They indicate that original recruitment to the panel was 44% and of that 44% only 36% (.44 x .36 = 16%) were available for study at the time of the September 11 surveys. Further reductions in the sample occurred subsequent waves. So, it is reasonable to conclude that this was no longer a representative sample.
The Caveat: Maybe It's All Normal. There are lots of other problems with the authors' interpretation of the results as indicating a distressed reaction extending across the whole American population. The authors leave for one sentence in the discussion section a concession that "Rather than considering be symptoms as evidence of psychiatric disorders per se, their presence in a substantial portion of individuals may in fact represent a normal reaction to an abnormal event." Probably this concession should have been highlighted earlier, because the authors did not even establish that the symptom levels they found were abnormally high, but then, the whole argument of the article would fall apart.
Beginning to Put Together an Alternative Explanation. Posttraumatic stress responses are generally understood as occurring in persons struggling with events that occurred in the past. However, in the months after September 11, 2001, the American population was facing considerable uncertainty and had legitimate concerns about whether this unthinkable event was ushering in period when other such events would continue to occur. They were not trapped in the past in their thinking, they were worried about the future. Thus, the response of the average American after 9/11 was quite different from the Iraqi war veteran who continues to be haunted stateside by things that he did or saw while in the field in Iraq, where he is confident he will never return. If anything, the average American after 9/11 was remarkably free of distress and in subsequent blogs, I will present evidence that this is the case.
As for the television viewing, people who were upset could have turned their televisions off. And such avoidance is a hallmark of posttraumatic stress reactions. If people remained glued to their television sets for hours, it was voluntary. Furthermore, they were not just getting exposure to the events of September 11, 2001, they were being repeatedly told by experts, including mental health professionals, that the events were traumatic. This is indoctrination, not just exposure. And maybe the persons who did the most viewing of their television were most symptomatic to begin with or were the most vulnerable to symptoms.
Stay tuned for my next blog post that will continue this discussion. The authors of these articles made it quite difficult to make comparisons with results that other investigators obtained in other samples. But a number of studies have accumulated and some of them are quite good and lacking in the problems that we have identified in these papers. The next blog will compare and contrast these two papers with the rest of the literature concerning reactions to September 11, 2001.
