When the Revolution Came for Amy Cuddy
It occurs to me that the replication crisis largely came into being because scientists are rewarded for publishing papers that demonstrate some effect and not for publishing papers that fail to demonstrate anything, and so the right fix would have been to fix the broken incentives. But apparently (if this article is to be believed) no such change in incentives has happened - the only thing is that there are now also incentives for demonstrating that some other study wasn&#x27;t replicable. Steadily doing <i>good science</i>, where good science is expected not to produce results much of the time, continues to have no support. Is it any wonder that the state of science seems a little better (at least we can admit the replication crisis now) but people like Cuddy feel like they have no future in the field, that having the misfortune of the one-in-twenty chance of getting a really interesting false positive at <i>p</i> = 0.05 means you either have to double down on the study being right after all or you have to quit the field? Are schools actually looking at past <i>p</i>-hacking while continuing to demand a huge publication record for hiring? This doesn&#x27;t seem like we&#x27;ve solved any fundamental problems.<p>(Also, as a side note, I&#x27;d fully expect that CS is at least as vulnerable to the same thing. McSherry et al.&#x27;s COST work is a good example of something that seems far <i>worse</i> than <i>p</i>-hacking: instead of having a clearly defined hypothesis and publishing the 1-in-20 papers that falsely state it&#x27;s true, as in other fields, there&#x27;s nothing clearly defined in scalable systems and 20 out of 20 people who build something excessively complicated can get a paper out of it.)