We don't know how to fix science (2021)
I see a lot of appeals to &quot;fund people, not projects&quot;, but the main issue I have with it is that I think it strongly biases people to select &quot;super stars&quot; who have impressive credentials &#x2F; come from high status organizations. Of course, there&#x27;s already a lot of &quot;rich get richer&quot; effects in science, as the article points out, but I think it would be nice to try to shift <i>away</i> from those things, not increase them. (Besides, isn&#x27;t the resounding message in academia these days supposed to be that assessing people is full of biases that we ought to avoid? Of course you always assess the people behind a grant application to some extent, but focusing on the project would seem to be better in terms of avoiding these biases.)<p>The other alternative of &quot;funding lotteries&quot; is more suspect. As the article summarizes, the argument for lotteries is:<p>&gt; <i>Advocates of lotteries make two key critiques: a) the current system forces researchers to spend a lot of time preparing grants; and b) peer reviewers cannot reliably identify “good” grant applications. They claim that a lottery system would reduce the time spent on review (because reviewers would mostly skim the proposals to check for minimal scientific robustness) as well as the time spent on preparing proposals (because there would be less of an incentive to meticulously craft proposals, given that no matter how detailed and well written they are, they are going to be chosen at random).</i><p>Of course (a) is true, but as the article suggests, the evidence for (b) is a bit shaky. Moreover, we should have strong priors against (b). To be sure, reviewing is not perfect, far from it! But, scientists must at some level be able to judge the future prospects of work, or else they wouldn&#x27;t be able to make any fruitful decisions about what research to conduct. So the primary plausible way I see that (b) could perhaps be true for a given pool of applications is if all of the ones in the pool pass some thresh-hold bar for quality that makes it hard to further distinguish between them. It&#x27;s not clear to me that that quality level would be maintained if we move to a lottery.