Replication Research Symposium

"You break it, you buy it" is the pottery barn rule that Srivastava (2012) suggested scientific to adopt with regards to publishing replications: They should publish replications for all original studies that they chose to publish. More than one decade later, journals do not adopt this rule, opening a gap for high-quality and researcher-led journals. I discuss how Replication Research is thought of a go-to journal for replication studies and why original journals should not be authors' first choice for publishing replications.

Repetition in research is often undervalued, dismissed as redundant or unoriginal. Yet, it plays a crucial role in verifying findings, refining methods, and building reliable knowledge. This talk explores different forms of repetitive research, its value across scientific disciplines, highlighting who benefits from it, and how it can drive innovation rather than hinder it. Embracing repetitive research can improve scientific credibility, support early-career researchers, and foster a more open and collaborative research culture.

What has started as a relatively simple idea 35 years ago – making scientific research available for free – has developed into a complex muddle of various variants of “freely available”, various stakeholders and their partly conflicting interests, various business models, and many aspects to consider when publishing open access.
The presentation aims to provide a brief overview of these aspects and thus situate the new “Replication Research” journal in an intricate environment.

We assessed all replications published in the top 100 psychology journals in the 10 years after the so-called replication crisis and investigated the nature of the replications, and the reasons provided for conducting the replications. We found that the replication rate is only half a percent across journals and that a predominantly novelty-oriented research culture even pervades the replication landscape.

Replication studies can take many forms (e.g., direct and conceptual replications). In my talk, I will present the replication study taxonomy that we developed to provide orientation for researchers, reviewers, and editors. Our taxonomy explicates that different types of replication studies serve different purposes and functions in the process of knowledge generation. Thus, the different replication studies also have different places within a systematic and cumulative research process.

Reproducibility is a foundational pillar of credible scientific research, yet it remains undervalued in many empirical fields. In this talk, I argue that reproducibility—verifying that original results can be precisely regenerated from the authors' own data and methods— and robustness analysis are not only feasible but cost-effective and scalable across disciplines. I describe how routine reproducibility checks can significantly enhance research quality, help identify inadvertent errors, and strengthen the credibility of scientific findings. Furthermore, I propose that reproducibility should perhaps be a mandatory step before resources are allocated to replication efforts using new data. Drawing on examples from economics, political science, psychology and related disciplines, I make the case that fostering a culture of reproducibility benefits the entire scientific ecosystem.

Concerns about the replicability of neuroimaging findings in mental health research have grown in recent years. Factors thought to undermine replicability—such as small effect sizes, limited sample sizes, and a high number of researcher degrees of freedom—are particularly pronounced in this field. Yet, direct replication studies remain extremely rare, and the true state of replicability is largely unknown. The Replicability of Findings in Neuroimaging in Major Depressive Disorder (ReFiNe-MDD) project addresses this gap by conducting 60 replication attempts of semi-randomly selected published findings on gray matter correlates of depression-related variables. To this end we utilize large-scale MRI cohort datasets comprising over 6,000 participants from both clinical and non-clinical samples, covering a broad range of psychological, biological, and lifestyle measures. In this talk, I will present the study design and pilot results from the ReFiNe-MDD project and discuss field-specific challenges we have encountered in replicating neuroimaging findings, related to preregistration, open data, and the definition of replication success in three-dimensional brain results.

The ZBW - Leibniz Information Centre for Economics has created an Open Access replication journal in 2017 and has been managing it since. The Journal of Comments and Replications in Economics (JCRE) and its predecessor (IREE) have in total published 41 articles so far. While this is not a very large number, it is a non-negligible share of total comments and replications published in economics journals. This talk will describe the strategy, editorial roles, processes, policies and content of JCRE. It will highlight the journal's role within the discipline of economics and discuss options for the future.

The FORRT Replication Hub aims to enable scientists to conduct high-quality reproductions and replications. It will serve as a comprehensive resource for research planning, reporting, publication, and teaching.

This discussion will revolve around two main themes:
- The high quality standards that we aim for with R2 will require considerable resources. How do we prevent lengthy review and reproducibility check times while still guaranteeing rigor?
- The core ideal of the journal is openness, but does this refer rather to the mandate of open data or to the inclusiveness and understanding brought forward to somebody who could risk their career in publishing a dataset that took years to create?

During the Friday morning hackathons, participants will create or revise R2 guidelines about 'article types' or 'TOP guidelines', incorporating the products of our previous discussions.

Please note that online participants may not be able to take part in the hackathons.

In this hackathon, we will review the R2 article types and make suggestions for revisions for the respective part of the R2 website (https://replicationresearch.org).

In this hackathon, we will review the R2 TOP Guidelines and make suggestions for revisions for the respective part of the R2 website (https://replicationresearch.org).

Each hackathon group presents their results in a 10-15 minute talk and gathers feedback from the plenum.

Concluding the Replication Research Symposium, I will summarize the efforts that have led up to its foundation and announce the action plan for the launch and starting-phase. Finally, we will reveal the official launch date of Replication Research.