The problem with p-values
<i>Take the proposition that the Earth goes round the Sun. It either does or it doesn’t, so it’s hard to see how we could pick a probability for this statement.</i><p>Sometimes I fail to follow the distinctions made in certain strains of classical statistics.  How is his &quot;conundrum&quot;  different than this one? &quot;Roll a 6 sided die whose result we cannot see.  Take the proposition that the top shows a 6. It either does or it doesn’t, so it’s hard to see how we could pick a probability for this statement.&quot;<p><i>What matters to a scientific observer is how often you’ll be wrong if you claim that an effect is real, rather than being merely random.</i><p>I think &quot;scientific observer&quot; may mean statistician here.<p>For the scientist, what should matter is the probability that the claimed effect is real -- period.  That is, unlike the statistician, the scientist isn&#x27;t (shouldn&#x27;t be) allowed to blame &quot;modeling error&quot; when it turns out that the measurements are biased, the samples are correlated, or the effects are nonlinear.  False assumptions that &quot;randomness&quot; is the only (or main) danger can lead to unrealistic error bars and unwarranted confidence in the effectiveness of flawed models.