Loss aversion is not supported by the evidence
Hmm...So here is an interesting thought experiment. Suppose you take a person with some appreciable intelligence (at least average) but no particular knowledge about a certain topic. In this instance, we'll let that topic be psycology.Now we present this person with an unfortunate dilemma. For a particular hypothesis, they observe a significant amount of peer reviewed literature asserting empirical evidence in the affirmative. They don't have any real familiarity with any individual papers, but they can understand that there's an established consensus.On the other hand they are given an article like this one, which mounts a critical refutation of all the established literature. Furthermore, this lone paper is presented to our stalwart examiner amidst the zeitgeist of a reproducibility crisis. Thus we have a mountain of peer reviewed but undigestible evidence on one side, and one readily digestible paper on the other which specifically rebuts the mountain of evidence.The fundamental dilemma is this: how should this person examine the available evidence to maximize their chances of coming to the correct conclusion? Should they abstain from trying to discern the truth of the matter, and strike out any opinion they could have as unqualified? Should they read through the most comprehensive surveys of available evidence to come to a full understanding? Should they take the new, contrarian paper at face value?There are a few dimensions here which (from my view) make the dilemma nearly intractable unless you 1) abstain from an opinion, or 2) become a subject matter expert. The cloud of the reproducibility crisis is a first dimension of uncertainty - not only does that muddy the waters for existig research, but we also have to be careful not take contradictory research at face value simply because it's contrarian. For another, we have to figure out how to weight the reliability of evidence against our own time. It's tempting to discount the evidence of existing research if an especially compelling critique is released, because we can more easily read it and follow its arguments.It seems like this is enormously difficulty all around. How do we rate the critical correctness of differing amounts of conflicting literature published under academic uncertainty?