While I agree with the bulk of what you said, I do think it&#x27;s important to understand that the difference with the Open Science Collaboration isn&#x27;t really what you said. In my opinion it may be worse.<p>Being a (somewhat disillusioned) economist, I&#x27;ve read some of these papers but certainly not all of them. What I can tell you is that there isn&#x27;t an &quot;experiment&quot; to replicate. These papers seem to be mostly or entirely computational macroeconomics and econometrics. In stuff like this, they design a model (a simple example is the real business cycle model), pump random data into it and see how different changes effect the model (i.e. the relationship between the volatility of unemployment with the volatility of output) and do those relationships match what we see in the data?<p>The replication as you define it (and I agree with the definition), should be pumping new random data into the model and still yielding the same results. However, it still leaves a few big issues. Such as, are these relationships really in the data? Because some of those relationships change depending on the time frame. So it the results may actually explain what occurred, but they shouldn&#x27;t be used to explain what will occur later.<p>For reproduction and re-analysis, for this research, they probably need to go together. If we&#x27;ve defined a mathematical model, then it should be possible to program that model across platforms and software and still yield consistent results with different sources of random input data. And for verification, I think this is really important. Because I know I&#x27;ve messed up my programs before and gotten completely reasonable output that turned out to be incorrect. The authors didn&#x27;t exactly describe how much they verified the programs were doing what they were supposed to do.<p>Honestly, I don&#x27;t know that I can make myself care too much about the output results from the model until we can agree on what things are important for the model to show in the past, present, and future. And in academic economics, these important characteristics are almost canon and untouchable.