Overcoming Barriers to Computational Reproducibility

Computational reproducibility, the possibility for independent researchers to exactly reproduce published empirical results, is fundamental to science. Despite its importance, the proportion of research articles aiming for reproducibility remains low and uneven across disciplines. Barriers include a perceived lack of incentives for researchers and journals, practical challenges in preparing reproducible materials, and the absence of harmonised standards of reproducibility processes and requirements by journals. Existing guidance is often highly technical, reaching mainly those already engaged with reproducible research. In this paper, we first synthesize evidence on the benefits of reproducibility for both authors and journals. Drawing on our extensive experience in reproducibility checking at various journals, we then put forward concise, pragmatic guidelines for creating reproducible analyses across disciplines. We further review current reproducibility policies of selected journals, illustrating the substantial heterogeneity in requirements and procedures. Motivated by the latter, we propose conceptual foundations for a harmonised multi-tier system of reproducibility standards that could support transparent, consistent assessment across journals and research communities. Our goal as journal (reproducibility) editors and contributors to the MaRDI initiative is to encourage broader adoption of reproducibility practices, in particular by lowering practical barriers for authors and journals.

💡 Research Summary

**
The paper “Overcoming Barriers to Computational Reproducibility” addresses the persistent low adoption of reproducible practices across scientific disciplines and proposes concrete, actionable solutions for both authors and journals. After defining computational reproducibility as the ability for independent researchers to exactly reproduce the final results of an analysis—including data, code, and software environment—the authors identify three core barriers: (1) insufficient incentives for researchers and journals, (2) practical burdens associated with preparing and sharing reproducible materials, and (3) lack of coordinated, harmonised institutional policies.

The authors first synthesize evidence on the benefits of reproducibility for individual researchers and for journals. For researchers, reproducible workflows improve code quality, facilitate future reuse, reduce the risk of undetected errors, and can enhance career prospects through increased citations and collaborations. For journals, reproducibility enhances transparency, strengthens peer‑review integrity, and can become a differentiating feature that attracts high‑quality submissions.

To lower the practical burden on authors, the paper introduces a tiered checklist that distinguishes Minimal, Recommended, and Optimal levels of reproducible documentation. The Minimal tier requires clear licensing, provision of raw data (when permissible), source code, and a list of software packages with version numbers. The Recommended tier adds version‑control practices (e.g., Git), environment capture (Dockerfile, Conda environment files), and executable scripts that automate the analysis. The Optimal tier goes further by integrating continuous‑integration testing, independent reproducibility verification by a dedicated editor, and the issuance of a reproducibility badge. This graduated approach allows authors with varying expertise and resources to adopt reproducibility incrementally.

On the journal side, the authors propose a multi‑tier reproducibility standards framework. Tier 1 (basic) mandates code and data availability statements and a simple reproducibility declaration. Tier 2 (intermediate) requires systematic verification by a reproducibility editor or reviewer, adherence to metadata standards (e.g., CodeMeta, DDI), and a more detailed reproducibility report. Tier 3 (advanced) incorporates automated workflow validation, ongoing maintenance of reproducible artifacts, and the awarding of a formal reproducibility certification. The framework is intentionally flexible so that journals can select the tier that matches their editorial resources and disciplinary norms.

The paper surveys the reproducibility policies of a selection of journals, revealing substantial heterogeneity: some journals only request data deposition, others provide detailed checklists, and a few have dedicated reproducibility editors. The analysis shows that journals with explicit, structured policies and dedicated staff achieve higher rates of successful reproducibility checks and report higher author satisfaction.

Finally, the authors situate their proposals within the broader Mathematical Research Data Initiative (MaRDI). They plan pilot implementations of the tiered checklist and journal standards across participating journals, collecting feedback to refine the model and eventually contribute to international standards for reproducible research.

In summary, the manuscript offers a comprehensive diagnosis of why computational reproducibility remains limited, and it delivers pragmatic, scalable solutions: a step‑wise author checklist to demystify the technical work, and a tiered journal standards system to align incentives and institutional support. By lowering both individual and systemic barriers, the authors aim to foster a culture where reproducibility is recognized as a standard component of scientific publishing rather than an optional extra.