Replication Research Symposium

"You break it, you buy it" is the pottery barn rule that Srivastava (2012) suggested scientific to adopt with regards to publishing replications: They should publish replications for all original studies that they chose to publish. More than one decade later, journals do not adopt this rule, opening a gap for high-quality and researcher-led journals. I discuss how Replication Research is...

Repetition in research is often undervalued, dismissed as redundant or unoriginal. Yet, it plays a crucial role in verifying findings, refining methods, and building reliable knowledge. This talk explores different forms of repetitive research, its value across scientific disciplines, highlighting who benefits from it, and how it can drive innovation rather than hinder it. Embracing repetitive...

What has started as a relatively simple idea 35 years ago – making scientific research available for free – has developed into a complex muddle of various variants of “freely available”, various stakeholders and their partly conflicting interests, various business models, and many aspects to consider when publishing open access.
The presentation aims to provide a brief overview of these...

We assessed all replications published in the top 100 psychology journals in the 10 years after the so-called replication crisis and investigated the nature of the replications, and the reasons provided for conducting the replications. We found that the replication rate is only half a percent across journals and that a predominantly novelty-oriented research culture even pervades the...

Replication studies can take many forms (e.g., direct and conceptual replications). In my talk, I will present the replication study taxonomy that we developed to provide orientation for researchers, reviewers, and editors. Our taxonomy explicates that different types of replication studies serve different purposes and functions in the process of knowledge generation. Thus, the different...

Reproducibility is a foundational pillar of credible scientific research, yet it remains undervalued in many empirical fields. In this talk, I argue that reproducibility—verifying that original results can be precisely regenerated from the authors' own data and methods— and robustness analysis are not only feasible but cost-effective and scalable across disciplines. I describe how routine...

Concerns about the replicability of neuroimaging findings in mental health research have grown in recent years. Factors thought to undermine replicability—such as small effect sizes, limited sample sizes, and a high number of researcher degrees of freedom—are particularly pronounced in this field. Yet, direct replication studies remain extremely rare, and the true state of replicability is...

The ZBW - Leibniz Information Centre for Economics has created an Open Access replication journal in 2017 and has been managing it since. The Journal of Comments and Replications in Economics (JCRE) and its predecessor (IREE) have in total published 41 articles so far. While this is not a very large number, it is a non-negligible share of total comments and replications published in economics...

The FORRT Replication Hub aims to enable scientists to conduct high-quality reproductions and replications. It will serve as a comprehensive resource for research planning, reporting, publication, and teaching.

This discussion will revolve around two main themes:
- The high quality standards that we aim for with R2 will require considerable resources. How do we prevent lengthy review and reproducibility check times while still guaranteeing rigor?
- The core ideal of the journal is openness, but does this refer rather to the mandate of open data or to the inclusiveness and understanding brought...

During the Friday morning hackathons, participants will create or revise R2 guidelines about 'article types' or 'TOP guidelines', incorporating the products of our previous discussions.

Please note that online participants may not be able to take part in the hackathons.

In this hackathon, we will review the R2 article types and make suggestions for revisions for the respective part of the R2 website (https://replicationresearch.org).

In this hackathon, we will review the R2 TOP Guidelines and make suggestions for revisions for the respective part of the R2 website (https://replicationresearch.org).

Each hackathon group presents their results in a 10-15 minute talk and gathers feedback from the plenum.

Concluding the Replication Research Symposium, I will summarize the efforts that have led up to its foundation and announce the action plan for the launch and starting-phase. Finally, we will reveal the official launch date of Replication Research.

The reproducibility crisis has shifted research focus toward improving the methodological integrity of animal experiments. Contributing factors to replication failure are being addressed, prompting the reassessment of fundamental methodological principles. For instance, rigorous standardization may restrict inference space and compromise external validity. To counter this, heterogenization...

Single case research designs are widely used in special education and related fields to evaluate interventions. The most commonly used approach to analyze these graphs is visual analysis. Despite existing challenges such as low interrater reliability, systematic bias, or a lack of training for raters, this method is most frequently applied due to its perceived simplicity and quick...

When analyzing electroencephalographic (EEG) data, researchers face a maze of potential methods and analytical approaches. But how much do these choices actually influence the results? The EEGManyPipelines (EMP) project addressed this question through a large-scale collaborative effort to test the robustness of EEG findings across different analysis strategies. In this unprecedented study, a...

Most experimental studies in social psychology rely on dichotomous factor-level designs, testing only the direction of effects rather than their functional form. As a result, replication efforts often reproduce these limited designs, reinforcing a narrow approach to theory testing. We argue that replication efforts should move beyond dichotomous designs to systematically map the shape of...

In a multi-lab replication study, my colleagues and me (Brohmer et al., 2024, https://doi.org/10.5334/irsp.522) recently replicated a classic effect of gender-fair language (Stahlberg et al., 2001, https://doi.org/10.1026//0033-3042.52.3.131): Participants, who were prompted to name celebrities via gender-fair language (e.g., "Bitte nennen Sie drei Politiker*innen"; English: "Please name three...

In many disciplines, multiverse analysis gains popularity as an important tool to strengthen the openness and transparency of research. Yet, multiverse analysis does not seem to be a common method in applied psychology, if it is known at all. Our key goal is to introduce the topic of multiverse analysis to applied psychology and to provide detailed guidance on how to run a multiverse analysis....

Infant research is particularly prone to problems of generalizability, and replications have not been part of standard practice. The Manybabies consortium has been established to run high-powered multi-lab studies, thereby increasing the robustness of infant research findings. The Manybabies model proposes a radically collaborative, large-scale and open approach to research grounded in...

Reproducibility is a core element of the scientific method. In the Geosciences, knowledge derived from geodata is frequently communicated through maps, and the computational methods used to produce these maps vary in their ease of reproduction. We present the results of a study in which we attempted to reproduce the maps included in geoscientific publications. We collected 27 candidate papers,...

Relational mobility, a socioecological variable, reflects the degree to which individuals in a society have opportunities to form new relationships and leave old ones. Previous research suggests that higher levels of relational mobility are associated with greater self-disclosure as an adaptive strategy to maintain valuable social ties. However, the advent of COVID-19 pandemic has...

Title: Replication of the Chameleon Effect (Chartrand & Bargh, 1999, Study 1)

In 1999, Chartrand and Bargh first demonstrated that humans have a natural tendency to mimic one another. The Chameleon Effect, named after this phenomenon, refers to “nonconscious mimicry of the postures, mannerisms, facial expressions, and other behaviors of one’s interaction partners, such that one’s behavior...

Unsuccessful replications often lead to fierce debates between replicators and original authors. This paper investigates whether arguably impartial experts reach consensus on a famous yet unsettled replication debate about the seminal paper by Acemoglu, Johnson, and Robinson (2001) and the replication by Albouy (2012). We successfully recruited experts from the pool of scholars citing one of...