In July, we posted an article about our Methods Reporting badge, It addresses gaps in methodological descriptions that can impair a study’s reproducibility.
This piece focuses on the other assessment we recently rolled out - data reporting. It is as critical to reproducibility as it is to the appropriate interpretation of a study particularly studies that rely heavily on statistics.
Inadequate statistical reporting creates unreliable results
Reproducible findings demonstrate scientific validity
Replication is critical to confidence around the interpretation of study results. Researchers repeat experiments over multiple samples - or multiple experimental replicates - to ensure that their conclusions are reliable.
In short, researchers want to know that what they’re reporting is what’s really happening in the universe. Experimental results, reported with sufficient clarity, should help readers understand the robustness of a study's conclusions.
Poor data reporting is common
Unfortunately, the majority of studies are plagued by poor data reporting. A recent study found statistical analysis or reporting issues in up to 96% of papers. Many researchers who are experts in their fields may still not be adequately trained in statistics. Without this training, it can be difficult to meet reporting standards for statistical analyses. One study argues that transparency in statistical reporting should be a strict requirement for publishing. Publication guidelines need to include strict transparency of statistical reporting in order to be published.
Data presentation adds an extra layer of complexity to preparing a manuscript; the description of visual and representative data must also meet these reporting standards.
With the persistent pressure to publish often, it can be difficult to keep statistical reporting top of mind. However, the details of data analysis are essential. Without them, readers are left unable to interpret data. As a result, the conclusions of a study are unreliable.
Reporting standards are key
Although issues with statistical analysis are sometimes addressed at peer review, they are easy for reviewers to miss. Reviewers, who are typically subject-matter experts, focus on the conclusions of a manuscript and the impact of the results on their field.
As a result, they may overlook whether a specific p-value was reported or whether adjustments were made for multiple comparisons. These details are critical to understanding and reproducing a study.
When statistical reporting is addressed at peer review, the revisions necessary to improve the reporting can hold up the peer review process because re-review is needed.
The Data Reporting Badge signals that your study’s reporting is complete
The Research Square Data Reporting Badge aims to address issues in reporting standards at an early stage - before peer review is complete. An editor will perform an independent assessment of statistics reporting standards to help improve your manuscript. Your study will be clearer, more useful, and easier to review.
Our Data Reporting Badge addresses key areas including:
- Sample Size: Precise sample sizes and/or replicate numbers provided for each type of experiment
- In-laboratory experimental repetitions: The number of experimental repetitions with similar results provided for representative data
- Statistical Tests and Values: Reporting of statistical tests performed as well as exact statistical values
- Data Presentation: Complete information about error bars, center values, plot elements, and indicators of statistical significance
Upon submitting your manuscript for a Data Reporting Badge, our editorial team will assess your manuscript within 3-5 business days. You’ll receive a short report with guidance on the revisions that must be made in order to be compliant with the relevant standards.
Once your manuscript has been verified to meet these standards, the corresponding badge icon will be displayed on your public preprint. You’ll also receive a downloadable certificate that can be used when submitting to a journal to demonstrate the quality of your manuscript.
Summary
In order to replicate and reproduce results, data reporting should be accurate. An accurate description of the statistics helps readers understand the study’s conclusions.
Poor statistical reporting is found in the majority of research manuscripts because many researchers are not properly trained in statistics. Peer reviewers do not often address statistical reporting problems. They are mostly concerned with results and conclusions.
The Research Square Data Reporting Badge exists to tackle statistical reporting problems before the peer review process. Our editorial team performs an audit of your reporting quality to ensure your manuscript is ready to be published as a preprint.
References:
Diong J, Butler A, Gandevia S (2018) Poor statistical reporting, inadequate data presentation and spin persist despite editorial advice. PLoS ONE 13(8): e0202121
Gosselin, RD. Insufficient transparency of statistical reporting in preclinical research: a scoping review. Sci Rep 11, 3335
For further reading:
Building Trust in Information Through Standards and Best Practices
Guidelines for Reporting Statistical Methods and Results