The Phrase “No Evidence” Is a Red Flag for Bad Science Communication
&gt; Is there &quot;no evidence&quot; that using a parachute helps prevent injuries when jumping out of planes? This was the conclusion of a cute paper in the BMJ<p>The BMJ paper doesn&#x27;t say there was no evidence!<p>The paper explicitly says there was anecdotal evidence. Here&#x27;s the relevant quote from <a href="https:&#x2F;&#x2F;www.ncbi.nlm.nih.gov&#x2F;pmc&#x2F;articles&#x2F;PMC300808&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.ncbi.nlm.nih.gov&#x2F;pmc&#x2F;articles&#x2F;PMC300808&#x2F;</a> :<p>] The perception that parachutes are a successful intervention is based largely on anecdotal evidence. Observational data have shown that their use is associated with morbidity and mortality, due to both failure of the intervention1,2 and iatrogenic complications.3 In addition, “natural history” studies of free fall indicate that failure to take or deploy a parachute does not inevitably result in an adverse outcome.4<p>&gt; Their [the BMJ authors&#x27;] point was that &quot;evidence&quot; isn&#x27;t the same thing as &quot;peer-reviewed journal articles&quot;.<p>How does Astral Codex Ten draw this conclusion? The phrase &quot;peer-review&quot; doesn&#x27;t exist in the BMJ article.<p>The point of the paper is that observational studies should not be rejected out-of-hand. Observational studies can be and are peer-reviewed.<p>&gt; Is there &quot;no evidence&quot; for alien abductions?<p>Some of the circled headlines used &quot;no evidence&quot; or even &quot;&#x27;Exactly zero&#x27; evidence.&quot;<p>But others used &quot;no hard evidence&quot; and &quot;no clear evidence&quot;, which are higher bars.<p>So, yes, there is evidence of alien abductions, but quoting an article I found using Google Scholar, it&#x27;s well characterized as: &quot;With so little hard evidence to support anything having to do with aliens on Earth — in fact, essentially no evidence outside personal testimony&quot; - <a href="https:&#x2F;&#x2F;cpb-us-e1.wpmucdn.com&#x2F;journeys.dartmouth.edu&#x2F;dist&#x2F;a&#x2F;1604&#x2F;files&#x2F;2020&#x2F;12&#x2F;Psych-7-Essay-3.pdf" rel="nofollow">https:&#x2F;&#x2F;cpb-us-e1.wpmucdn.com&#x2F;journeys.dartmouth.edu&#x2F;dist&#x2F;a&#x2F;...</a><p>(It then points out other psychological mechanisms which might explain the evidence, ending &quot;Whether their stories are true or not, their trauma is very real, and it deserves to be treated as such.&quot;)<p>&gt; Is there &quot;no evidence&quot; homeopathy works?<p>Sure, there&#x27;s evidence. But there&#x27;s also counter-evidence.<p>Is there stronger evidence that homeopathy works than there there was evidence that COVID-19 could spread through the air-con system?<p>Probably.<p>So how is this example supposed to help make the author&#x27;s point?<p>&gt; Here we should reject journal articles because they disagree with informal evidence!<p>Q: What is the basis of the author&#x27;s informal evidence? A: &quot;a strong theoretical understanding&quot;. Q: Where did that strong theoretical understanding come from? A: Published research.<p>So the author is rejecting a journal article because it disagrees with other journal articles, after being watered through public perceptions.<p>That&#x27;s not informal evidence!<p>&gt; there is literally not a single piece of evidence anywhere in the world that anyone could use in favor of snake oil.<p>The author earlier accepted anecdotal evidence of alien abductions as a type of evidence.<p>How is the author so sure there is no anecdotal evidence that snake oil works?<p>It&#x27;s because of essays like these I avoid substack links.