Authors’ names have ‘astonishing’ influence on peer reviewers: study
Double-anonymous&#x2F;blind reviewing is completely standard in most areas of computer science, e.g. networking (SIGCOMM, NSDI, IMC, HotNets, MobiCom, MobiSys), systems (OSDI, SOSP, USENIX ATC, HotOS), security (Usenix Security, S&amp;P), machine learning (NeurIPS and ICML), graphics and HCI (SIGGRAPH and CHI), and at least some top-tier theory conferences (like FOCS).<p>So I think most computer scientists would agree with the article&#x27;s conclusion that double-anonymous reviewing, while flawed, is better than the alternatives. I don&#x27;t think I&#x27;ve done a non-anonymous submission in 10 years, and as a reviewer, usually I don&#x27;t have a strong guess about who wrote the paper (and when I think I know, often I turn out to be wrong). It&#x27;s a little annoying that this news article in Nature Magazine ignores the longstanding widespread prevalence of this practice in a conference-driven (but... we like to think important) academic discipline. :-)<p>But:
(1) I don&#x27;t think the big benefit of double-anonymous reviewing is that a lousy paper from a Nobel laureate (or, from CMU&#x2F;Berkeley&#x2F;MIT&#x2F;Stanford) isn&#x27;t let in unfairly. To me the big benefit seems to be that reviewers have to review every paper <i>as if</i> it might be from their friends or a famous person, and consequently have to review each paper with due care and try sincerely to understand its contribution, which maybe they wouldn&#x27;t do if they knew it&#x27;s from some random place&#x2F;author they&#x27;ve never heard of or don&#x27;t think highly of. I think the way it affects judgment may be more in equalizing time&#x2F;effort spent to read and understand a paper (and the generosity you give a paper because &quot;maybe&quot; it was written by your friend or somebody you respect), rather than a straight-up bias towards liking whatever the faculty at a famous university are writing about this year.<p>(2) While I do think it&#x27;s true, and a good thing, that double-anonymous reviewing helps &quot;marginalized groups of authors who often struggle to have their work see the world&quot; as the lead researcher says, we should probably acknowledge that authors are not the <i>only</i> beneficiaries of a scientific publication. The interests of the reader matter too -- the journal or conference has <i>some</i> duty to serve them. On the margin, maybe some readers would be more interested to learn what Albert Einstein is thinking about these days, or would like to see a well-balanced conference program that includes a good talk by a known-provocative speaker, instead of one more random (but adequate!) paper from a nobody. I&#x27;m not saying we should give a huge weight to this -- it&#x27;s fine to make people eat their vegetables, but, I don&#x27;t think we should act like scientific publication is only to give authors a line on their CV and the readers&#x27; preference is 100% irrelevant. A scientific journal shouldn&#x27;t <i>exclusively</i> serve the authors. (Other kinds of media care way too much about what the reader wants, e.g. Facebook giving you whatever it thinks will keep you clicking things on Facebook, but there is probably a happy medium somewhere.)<p>(3) The challenging frontier may be in grant submission and reviewing, where proposers are typically <i>not</i> anonymous to the reviewers, which surely leads to some biases. I have heard about government programs where they did use double-anonymous reviewing and it seemed weird to me. (Probably this is a situation where track record should matter, yet trying to summarize your own track record while remaining effectively anonymous seems really hard...)