An Elaborate Academic-Journal Hoax
Discussed earlier on HN: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=18127811" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=18127811</a><p>The hoaxers writeup is, I think, pretty misleading. Their work tended to get rejected from high-impact journals. They submitted multiple stories to some journals (notably, to <i>Hypatia</i>, which they brag about getting accepted in) and leave a clear impression that their most ridiculous work was what made it in, rather than a much more banal paper. They quote almost entirely favorable feedback --- including, in their earlier writeup, from papers that were ultimately rejected! They brag about papers that were ultimately rejected. They submit approximately the same paper to multiple journals and are rejected by all but the very most marginal. In at least one case, they cite the same reviewer multiple times, leaving the impression of a consensus.<p>If you don&#x27;t think that&#x27;s misleading, it&#x27;s worth knowing that Yascha Mounk, cited warmly in this piece, was clearly taken in, dazzled on Twitter by a rejected paper from the hoax.<p>What have they really uncovered?<p>1. Peer review isn&#x27;t replication and never could have been. Peer reviewers get, at best, a couple hours, unpaid; submitting researchers get many months, and are often grant-funded. The hoaxers took <i>ten months</i> to get a few papers accepted in marginal journals. The pop understanding of what peer review is is, simply, broken: peer review depends on good faith, and it was never realistic to believe otherwise.<p>2. There are lots of crappy journals. Didn&#x27;t you already know that? The market responds to the demand that professional academics publish, by creating venues that ensure professional academics can reliably publish. That&#x27;s why we have measures like &quot;impact factors&quot;.  This is not <i>at all</i> a phenomenon isolated to the social sciences.<p>If the authors even <i>acknowledged</i> these conclusions, they&#x27;d take all the excitement out of their hoax. We&#x27;d all nod our heads and say, &quot;of course&quot;. Instead, they pretend to have taken down something they&#x27;ve dubbed &quot;grievance studies&quot;, while pretending that they couldn&#x27;t have done the same thing to, say, a marginal economics journal.