Verified by 
Coronavirus Disease 2019
How to Best Approach COVID-19 Research
Reading the latest science on COVID? Here’s why you may need to be careful.
Posted August 21, 2020 Reviewed by Devon Frye
The COVID-19 pandemic has led many people to directly engage with biomedical research studies—often for the first time in their lives—in the search for answers, understanding, and hope.
The official guidelines provided by national, state, and local governments and health officials are critically important, but generally do not include much beyond rules, regulations, and recommendations. These in turn are informed by relevant epidemiological information such as case counts that are regularly reported by news outlets and can be directly accessed on academic and government websites. Finally, the swirling cesspool of dangerous conspiracy theories presented in various online forums can serve as a harmful distraction for those grasping for a feeling of control in these uncertain times.
But given the rapidly changing nature of the pandemic, diagnostic and therapeutic developments, and the ultimate hope for a suitable vaccine, biomedical research data is being accessed by the non-scientist—whether through clicking on links in news stories or by direct web searches.
Before discussing the benefits, and potential pitfalls, of engaging with primary research, it is important to consider how science gets done. Historically it was common to consider a divide between the “pure” basic science research conducted within academic institutions, and the applied work performed in biotech and pharmaceutical companies. Things like clinical trials where drugs, procedures, and technologies are tested for medical safety and efficacy have long been examples of collaborations between the private sector and hospitals and universities.
However, even before COVID-19 arrived these lines have become blurred largely beyond recognition. Spin-out companies from faculty research, incubator spaces owned by universities, and individuals with careers that cross the spectrum from academia to industry are all examples of the requirement that most biomedical research have some sort of direct application.
Things have gotten much more complex than the conventional model where an academic researcher has an idea, gets a grant, performs the work, publishes the results in a peer-reviewed journal, and then moves onto the next research project. Intellectual property considerations such as patents and licensing opportunities have become more standard, as have direct links between companies and universities. However, throughout this changing landscape, the long quoted axiom, “publish or perish,” still reigns.
Research papers often serve as the basis for patents, faculty advancement through the academic ranks generally requires regular well-received publication in highly regarded journals, and without peer review as a cornerstone of the vetting process, scientific results would not be openly and effectively scrutinized. However, this process is evolving and two interesting recent developments in the centuries-old process of publishing scientific data are:
- the rise of “open access” journals
- the creation of online repositories for unpublished manuscripts
Open access at its most fundamental simply means that nobody should have to pay to read the results of scientific studies. The general rationale is that if my tax dollars paid for the grant, I should be able to see the results without having to have a journal subscription or use my credit card number to get past a paywall on a website. Furthermore, by allowing anyone to access the paper you are leveling the playing field so that all researchers can compete equally, regardless of the budget of their university library. Importantly, in an effort to increase the availability of critical new data, many scientific journals that are not normally open access have made COVID-19 research papers freely available.
However, even before publication, researchers want to share their results. Websites such as bioRxiv, pronounced "bio-archive," and medRxiv, allow basic scientists and medical researchers, respectively, to get their data out without delay before peer review, revision, and publication. This increases visibility and serves the researcher, scientific community, and, especially now, journalists and the general public. Taken together with the rise of open access journals, these repositories for sharing so-called "preprints" serve to increase access to primary scientific data that was previously only in the hands of experts. However, this can be a double-edged sword.
It is often said that science is self-correcting. Research is generally methodical and involves repetition, where the researcher does the same thing over and over again to make sure they get the same result each time, and replication, where other researchers try to reproduce your experiment to see if it holds up when performed by another set of hands. It is not unusual for a contentious and groundbreaking result to be subject to significant skepticism, scrutiny, and follow-up confirmatory study, as it should be. However, with a global pandemic showing no signs of ending, there is a major drive to get as much information out as quickly as possible, and the normal guardrails inherent to the standard scientific method might understandably be loosened.
It is into this context then that people now might find themselves reading an article online in the press, even reported by the most reputable media outlets, and then clicking a link that brings them to a newly published paper, or recently posted pre-print, and coming face-to-face with research data, potentially for the first time. However, even for scientists that are experts in the topic of a specific research study, the devil is very much in the details. Serving as a peer reviewer for a scientific journal is extremely time consuming and takes countless hours of work going through different drafts of a manuscript in great detail. Furthermore, once a study is published, researchers will often analyze particular papers in what is referred to as a "journal club"—which is a bit like a book club, but usually with less wine.
Here groups of researchers, faculty members, and trainees go through a paper and dissect every aspect from the rationale to the methodology and ultimately the data analysis and conclusions. Again this is very time-consuming work, even for experts. So how can an individual, who is not a trained scientist, possibly hope to truly digest the ramifications of a scientific study? This is particularly true when the popular science press, generally the source of the link to the paper, in an understandable attempt to abstract the work to create a fundamental take-home message useful and interesting to the reader, might not place the work in the appropriate context, or view it with sufficient skepticism.
Take, for example, the recently suggested link between blood type and COVID-19. Ranging from clinical correlations to genetic analyses, including both published papers and preprints that have not yet been peer-reviewed, the seemingly simple investigation of a possible role of blood type in an individual’s risk has begun to spin wildly beyond easy understanding. Some studies have suggested a link, while others have challenged this view. Overall this seeming confusion is likely due to the drive to draw broad conclusions from studies that are by design very specific and focused in scope.
The important questions to ask include: What is actually being measured? The risk of becoming infected is not the same thing as the rate of becoming hospitalized, admitted to the ICU, or mortality. Similarly, what is the comparison being made? Are the researchers looking at two similar populations, both that are infected with the virus, to determine differences in progression, or comparing an infected study group with an uninfected control?
In the case of clinical trials, it can be relatively straightforward to draw direct causative conclusions if the treated group recovers and the control group does not. However, many research studies are correlative, and the results aren’t always definitive or pointing to a direct mechanism. It can be very difficult to account for all the differences between two populations, and so it can be very easy to assume that alternative outcomes are actually the result of a pet hypothesis or easily understood mechanism.
Unfortunately, only time can help clarify these types of complex issues, both for carefully conducting individual studies, and the analysis and follow-up work required for confirmation and understanding. However, time is certainly the one thing we don’t have on our side.
So what can be done? The most important thing is to recognize that good science takes patience, and anecdotal evidence should not be taken as proof. Correlation is not the same as causation, and the recommendations and decisions made by groups such as the WHO, CDC, FDA, AMA and others regarding effective infection control measures and the standard of care include detailed analyses of all the information available, not simply the highly publicized results of individual reports. Looking at the raw data can be empowering and informative, but groups of experts assessing all available information is how an overall understanding can emerge that will result in medical advances and well-informed policy decisions.
