Beware the Man of Many Studies
Great article. This part especially rings true to me: "If researchers have an ideological bent, a meta-analytic null may just be an expression of the typical sentiments of researchers".<p>When I was in academia, it was increasingly the case that my peers thought of research less as a way to determine the truth, but just as a method to influence policy and public opinion. If we thought something was 80% likely to be true, there was pressure to "close ranks" and pretend as though it was 100% true, and to avoid publishing anything that contradicted it. It is also well known that papers that support certain "sides" tend to be easier to publish (and in higher ranked journals), plus can yield more media attention. See for example, this fraud in sociology - <a href="https://en.wikipedia.org/wiki/When_contact_changes_minds" rel="nofollow">https://en.wikipedia.org/wiki/When_contact_changes_minds</a>.<p>This may be better in the natural sciences, but in social science you should not trust any paper unless you read through and fully understand the methodology. Any non experimental results has so much wiggle room in the modeling methodology that it's easy to generate any result you want.    The actual percentage of papers with credible results is very low, much lower than laypeople think.