Publication Type

Journal Article

Publication Date

9-2016

Abstract

This crowdsourced project introduces a collaborative approach to improving the reproducibility of scientific research, in which findings are replicated in qualified independent laboratories before (rather than after) they are published. Our goal is to establish a non-adversarial replication process with highly informative final results. To illustrate the Pre-Publication Independent Replication (PPIR) approach, 25 research groups conducted replications of all ten moral judgment effects which the last author and his collaborators had “in the pipeline” as of August 2014. Six findings replicated according to all replication criteria, one finding replicated but with a significantly smaller effect size than the original, one finding replicated consistently in the original culture but not outside of it, and two findings failed to find support. In total, 40% of the original findings failed at least one major replication criterion. Potential ways to implement and incentivize pre-publication independent replication on a large scale are discussed.

Keywords

Crowdsourcing science, Replication, Reproducibility, Research transparency, Methodology, Meta-science

Discipline

Management Sciences and Quantitative Methods

Research Areas

Organisational Behaviour and Human Resources

Publication

Journal of Experimental Social Psychology

Volume

66

First Page

55

Last Page

67

ISSN

0022-1031

Identifier

10.1016/j.jesp.2015.10.001

Publisher

Elsevier

Creative Commons License

Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.

Additional URL

http://doi.org/10.1016/j.jesp.2015.10.001

Share

COinS