I have actually been on grant committees before and the basic problem is you only have enough funding for 10% to 15% of the proposals. Almost all of the applications are really good and from good people. The reason for this is it is so much work (at least 6 to 8 weeks full time) to write a grant application that only the really good people put them in. On top of this the universities will “pre-review” all the grant applications internally before they go in to make sure that all the weak proposal are weeded out and all the obvious flaws removed.<p>The end of this process is that almost every grant is really, really strong making it impossible to rank them consistently. The committee ends up ranking on trivia, or in the worse case on “old-boy” connections. We are asking these committees to do something that we know is impossible.<p>If we can’t rank grant applications by quality then lets stop pretending we can. Just decide if they are strong and then put all the strong ones into a lottery and fund as many as we can.