&gt;&quot;success criteria down the road (spawning high-profile publications, generating citations, coming in under-budget, whatever...)&quot;<p>I wouldn&#x27;t handwave this aspect. Creating incentives to optimize the wrong thing can be worse than using an arbitrary filter. Your first two sound like they encourage hype, popularity contests, and quantity over quality.<p>How about something combining reproducibility of analysis, availability&#x2F;sharing of the data, precision of theoretical prediction,  and consistency of quantitative estimates from independent replications?