Evidence-based medicine (EBM) is “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.” EBM was founded in the 1990s at McMaster University. A system was established to integrate the best available external evidence from systematic research, available resources (clinical expertise, personnel, facilities, funding), and patient preferences to arrive at empirically formed clinical conclusions.
EBM is a stringent and rigorous form of clinical research that follows structured unbiased research methods to objectively assess the overall state of evidence around healthcare topics and identify if biases are present in the literature that would deviate the conclusions away from the truth.
The basis of EBM
The foundation of EBM is formed through conducting systematic reviews/meta-analyses (SR/MA), which are studies that assess the overall body of evidence around a topic and/or intervention to examine associations, safety, and/or efficacy and effectiveness.
Researchers conduct SR/MAs by summarizing the literature through the following steps: aggregating the totality of the literature around a subject; narratively and quantitatively pooling the results from each study to examine the relationships between individual studies; assessing the risk of bias of each study; and arriving at conclusions regarding the direction (beneficial or harmful) and significance (if the effects are statistically significant and clinically meaningful) of the results.
Conducting systematic reviews, meta-analyses
To conduct a SR/MA, multiple bibliographic databases, trial registries, conference abstracts, and the grey literature are searched to identify all relevant articles around the investigated topic. Database searching typically returns hundreds to thousands of articles.
The retrieved literature is examined to find appropriate articles for inclusion in the SR/MA that have characteristics prespecified by the researchers. Despite the extensiveness of the search process, the goal to identify the maximum amount of topic-related content is undermined when trial results have never been published. Examinations comparing the amount of published versus unpublished literature demonstrate:
- Out of 807 clinical trials, 48% were unpublished.
- Out of 594 clinical trials, 50% were unpublished.
- Out of 631 clinical trials funded by the NIH, less than half were published after 30 months of a study’s completion and one-third remained unpublished after a median of 51 months.
- Out of 13,327 clinical trials posted in ClinicalTrials.gov, only 38.3% reported results.
- Out of 29,729 abstracts that presented randomized controlled clinical trial results, 37% were never disseminated in a full publication, and those with positive results were 30% more likely to be published.
These results demonstrate that a significant proportion of the literature across the biomedical spectrum is never published. When literature remains unpublished, or specific types of literature are published in favor of others, reporting biases are introduced. For example, compared to studies demonstrating statistically nonsignificant or negative results, studies with positive or statistically significant results have 2.2-4.7 higher odds of being published; are published faster (in 4-5 years versus 6-8 years); and are cited 1.6 times more from higher impact factor journals.
With publications acting as a career-driving mechanism for researchers to demonstrate impact, it is unfavorable for researchers to suppress results. These results imply that publication bias may be fueled more by publishers’ preferences rather than researchers’ selective reporting.
Since researchers are at the mercy of publishers to choose the trials that are “worthy” of publication, what mechanism can researchers use to disseminate their results?
To decrease publication bias and ensure that all research becomes accessible, researchers can post their work in preprint repositories that offer a near-guaranteed mechanism for sharing research. When researchers disseminate their work in preprint repositories, they protect their work from being scooped and ensure their research will be discoverable by EBM researchers regardless of the decisions (accept or reject) of journal editors.
It's critical that all available literature can be identified through database searching because failure to include available trials in SR/MAs, assuming they meet the researchers’ pre-specified inclusion criteria, may lead to under- or overestimated results, incorrect conclusions, and inappropriate recommendations.
The value of indexing
As preprint dissemination and reception grows, preprints will increasingly be indexed in databases that EBM researchers rely upon to identify articles for inclusion in SR/MAs, such as MEDLINE, CENTRAL, Embase, and Science Citation Index. In 2013, an average of 233 biomedical preprints were posted per month, representing 0.19% of all published biomedical literature that year. As of June 2020, 9,911 biomedical preprints per month were posted, representing 8.08% of all published biomedical literature in 2020. In 2020, The National Institutes of Health (NIH) initiated the NIH Preprint Pilot, a project aiming to “make preprints resulting from research funded by the National Institutes of Health (NIH) available via PubMed Central (PMC) and, by extension, PubMed.”
Through the pilot, preprints are searchable via PMC and PubMed, made available by leading preprint repositories, such as Research Square, bioRxiv, medRxiv, ChemRxiv, arXiv, and SSRN. Additionally, Europe PMC indexes over 37,000 preprints from a wider selection of databases than the United States-based PMC, including Research Square, bioRxiv, PeerJ Preprints, ChemRxiv, F1000Research, Gates Open Research, Wellcome Open Res, HRB Open Research, AAS Open Research, and MNI Open Research.
Preprints and EBM
While preprints are drafts of finalized research reports, they have significant potential to positively impact EBM, and thereby, the wider healthcare community. EBM does not discriminate against the publication status of a manuscript. What is important is that EBM researchers have access to the data, an output that preprints excel at providing to the scientific community.
Healthcare is served through the discovery and dissemination of learned knowledge, derived from research and how it is interpreted and translated to practice. Preprints are the key for researchers to open doors locked by publishers that may stymie the knowledge translation process and hinder the full and accurate application of research into practice.
Ultimately, the union of preprints with EBM has the potential to enhance the accuracy and comprehensiveness of clinical recommendations. Preprints lead the future of scientific publishing, and EBM is a domain that will benefit from this unbiased researcher-driven dissemination mechanism.