The incentives of academia naturally select for less reliable results (2016)
The proposed &quot;solutions&quot; of pre-registration and increase sample size are not going to work because they are not fixing the underlying problem of NHST use (where a researcher tests a nil null hypothesis of zero effect and then concludes their favorite explanation is correct if they reject this null hypothesis).<p>The solution is to go back to the old way of science (pre-1940):<p>1) Collect data and figure out what is repeatable and consistent (eg come up with &quot;natural laws&quot;).<p>2) Come up with theories that explain the observations from #1 that also make <i>precise</i> predictions about some different type of data.<p>Reward researchers who figure out methods that lead to reliable, repeatable observations and those that make precise predictions (if your theory predicts nothing more than a positive&#x2F;negative correlation between two variables it is too vague).