The Hawthorne Effect And The Overestimation of Treatment Effectiveness
Science struggles with psychology.
Posted Nov 27, 2010
The Hawthorne effect was originally defined in an industrial setting: The Hawthorne Works Plant was studied by researchers at Harvard University who were trying to determine the relationship between productivity and the work environment.
It was found that there was an increase in worker productivity produced by the psychological stimulus of being singled out and made to feel important, through the attention of the research team. The Hawthorne effect has become a term referring to the tendency of some people to work harder and perform better when they are participants in an experiment; behavior is changed due to the attention subjects receive from the researchers, rather than because of any manipulation in dependent variables.
In the clinical trial setting, the effect may be defined as the additional clinical response that results from increased attention provided by participation in the clinical trial. And in fact, improvement in rheumatoid arthritis clinical trials is often greater than improvement seen in the clinic. Interestingly, follow-up clinical trials consistently show durability of response.
A recent study presented at this year's American College of Rheumatology annual scientific meeting in Atlanta examined 264 rheumatoid arthritis patients, the goal to assess the effect, if any, of the Hawthorne effect, and whether the results of clinical trials are upwardly biased by this phenomenon. The researchers theorized that the Hawthorne effect could result in improvement in rheumatoid arthritis through two mechanisms:
1. It could result in true improvement, as in the Hawthorne effect of observed productivity in the factory.
2. It could result in reported, yet false improvement, expressed perhaps in an overly optimistic study subject questionnaire.
If the Hawthorne effect were indeed present in this setting, it would be expected that patients would not fare as well in the subsequent clinical setting, in the absence of the more intensive clinical trial research attention.
As for the results, it was found that all study measures improved during the trial, and worsened thereafter. These measures included assessments of activities of daily living, pain and fatigue. The Health Assessment Questionnaire score improved by 41.3% during the trial, but only by 16.5% when the endpoint was the post-trial result. Likewise, pain improved by 51.7% during the trial, and by 39.7% at the post-trial time point; the respective percentages for fatigue were 45.6% and 24.6%.
By way of conclusion the authors of the study found that significant improvement noted in the clinical trial disappeared upon entry to a non-sponsored follow-up study. Such changes are apparently due to the Hawthorne effect. It would indeed seem that the values of patient reported rheumatoid arthritis outcome variables in clinical trials are upwardly biased; treatment effect is actually less than is observed and subsequently disseminated in articles in journals and interviews in the media.
Science thus struggles with psychology, although the latter can give more than even science might dare expect.