The Skeptical Sleuth

Applying a healthy dose of skepticism to new findings about health and psychology.

Investigating the Accuracy of Abstracts: An Introduction

Bad abstracts can kill, but mostly just spread bad science.

This blog introduces a series of occasional blogs that will investigate the accuracy of abstracts of journal articles. As we will see in the series, abstracts are often quite misleading and tend to exaggerate the strength and importance of findings.

A PLoS Medicine article noted:

Arthur Amman, President of Global Strategies for HIV Prevention, tells this story: “I recently met a physician from southern Africa, engaged in perinatal HIV prevention, whose primary access to information was abstracts posted on the Internet. Based on a single abstract, they had altered their perinatal HIV prevention program from an effective therapy to one with lesser efficacy. Had they read the full text article they would have undoubtedly realized that the study results were based on short-term follow-up, a small pivotal group, incomplete data, and unlikely to be applicable to their country situation. Their decision to alter treatment based solely on the abstract's conclusions may have resulted in increased perinatal HIV transmission.”

Find a Therapist

Search for a mental health professional near you.

Abstracts need to be clearly and accurately written because many readers do not read further when doing an electronic search of the literature. Readers decide whether to download an article on the basis of the abstract, and even if they download the article, the abstract becomes the key information recalled about the article. Moreover, with some of the highest impact medical journals still behind pay walls, healthcare professionals who don't have costly subscriptions or access through a University libraries, may depend solely on abstracts, like the unfortunate South African physician described in the quote.

Inaccurate abstracts can perpetuate myths and misinformation and contaminate other sources. Many careless authors rely entirely on the abstract for the information that is cited in their manuscripts. That frequently proves a mistake, but their mistake gets perpetuated in subsequent articles whose authors don't check their sources either.

The task of writing an abstract often becomes an occasion for sacrificing accuracy to "best foot forward" and hype. Increasingly, editors send only a portion of the manuscripts they receive out for review and they may not even read full manuscripts themselves, but rely on the title, abstract, and cover letter for a decision whether to send it out for review or simply to return to the authors with a simple statement "of insufficient interest to our readers.” Given a strong confirmatory bias to what gets published—a strong preference for only publishing positive findings—authors come to believe it is important that abstracts highlight the strength and significance of findings, even if that involves some distortion. As we will see in the series of blog posts, authors often go too far, but if they succeed in publishing an abstract that sacrifices accuracy to spin and bias, it is a failure in the responsibility of the journal to its readers and the field, as well as a failure of the author.

But even when abstracts do not involve deliberate distortions, they too often provide insufficient information or leave readers with misimpressions.

There has been a lot of confusion about what should be said in an abstract and so a movement gained momentum to structure abstracts and educate authors as to what they need to report. While some journals now require structured abstracts fitting a particular format, many do not, but it is still best to proceed with a sense of what needs to be there. And it is particularly important to recognize just how crucial an abstract can be for who access your article and will they remember about it.

When I am writing a manuscript, I write the abstract early and use it to structure the rest of my writing, jumping around within that structure when I get stuck in any particular point in writing the manuscript. But then I go back and rewrite the abstract again and again so that it fits the emerging message of the manuscript. I take the writing of the abstract quite important in terms of the way it will be depended upon for conveying the message of the article. I choose my words carefully, cognizant of the tight limits on number of words that require choosing to say one thing at the expense of being able to say other things.

One type of manuscript, the reporting of randomized clinical trials evaluating medications and other interventions, has more specific and enforced requirements for an abstract. Standards have become part of CONSORT (Consolidated Standards for Reporting Clinical Trials).

A published checklist is available for evaluating the adequacy of abstracts, and there is also an article that explains why particular details are important in an abstract. These tools can guide authors in making decisions about what to include, but serious inadequacies in published abstracts remain, and poor reporting of a study often hides the study’s shortcomings and weakness of effects.  Furthermore, the checklist merely provides a guide as to what needs be reported and does not itself evaluate its accuracy. Items on the checklist are ticked off if the required information is present in the abstract and not in terms of whether it is accurate. But if the checklist can certainly be used to assess accuracy and serves as a useful guide comparing the abstract with what is reported in the body of the article.

You might want to familiarize yourself with these tools in anticipation of the series that will be beginning shortly. My next post will concern the abstract of the article appearing in Annals of Behavioral Medicine in terms of whether it adequately communicates the findings and significance of the study. We will use the abstract to cultivate skepticism is found in the scientific journals and to sharpen sleuthing skills, to discover how to compare an abstract versus the information contained in an article.

Nina Heinrichs, Tanja Zimmermannm Brigit Huber, Peter Hershbach, Daniel Russell et al. Cancer Distress Reduction with a Couple-Based Skills Training: a Randomized Controlled Trial.  Annals of Behavioral Medicine. (2012) 43:229-252.

Because the article is behind a paywall that charges a fee if you cannot access the article through your library, you might simply want to write to the author for a PDF at nina.heinrichs@uni-bielefeld.de

For a succinct, excellent guide to writing abstracts, including how many words to devote to each section, go to this link. But, if you're writing an abstract for a clinical trial, you should also use this guide in combination with the CONSORT checklist. Finally, it is always a good idea to check the website of the journal to which you are submitting your manuscript, because requirements vary, and particularly the number of words allowed.

Jim Coyne, Ph.D., is a clinical health psychologist and Professor in the Department of Psychiatry at the University of Pennsylvania. more...

Subscribe to The Skeptical Sleuth

Current Issue

Just Say It

When and how should we open up to loved ones?