Samuel Pawel|Leonhard Held
Probabilistic forecasting of replication studies
University of Zurich|University of Zurich
throughout last decade socalled replication crisis stimulated many researchers conduct largescale replication projects data four projects computed probabilistic forecasts replication outcomes evaluated regarding discrimination calibration sharpness novel model take account inflation heterogeneity effects used predicted effect estimate replication study good performance two four data sets two data sets predictive performance still substantially improved compared naive model consider inflation heterogeneity effects results suggest many estimates original studies inflated possibly caused publication bias questionable research practices also degree heterogeneity original replication effects expected moreover results indicate use statistical significance criterion replication success may questionable since predictive viewpoint nonsignificant replication results often compatible significant results original study developed statistical methods well data sets available r package replicationsuccess