Using loaded dice to cheat at Settlers of Catan, and p-values to avoid suspicion
Very nice hack and analysis thereof.<p>I do take issue with this statement, though, in particular the part highlighted (by me):<p>&gt; &quot;it’s impossible for your opponents to scientifically prove that you’re cheating. <i>This impossibility is due to methodological defects in the current state of scientific practice</i>, and we’ll highlight some ongoing work to fix these defects.&quot;<p>The impossibility is not due to methodological defects in the current state of scientific practice. It&#x27;s intrinsic to the world. If you cheat only a bit, and only rarely, it might be impossible to detect it statistically.<p>Of course, there are problems with the current state of scientific practice and the mindless application of statistics (particularly the 5% significance level). The replicability crisis in psychology comes to mind.<p>However, I think statistics is unfairly maligned here. Statistics mostly delivers on its promise, it just promises less than what some people think.<p>The world is messy, and statistics can help us to make sense of it, but it can only do so much. In particular, it&#x27;s not magic. You need many observations to make statistical statements, and more observations to make more precise statements (or statements at a higher significance level). That&#x27;s intrinsic to how the world is. It&#x27;s not a problem with &quot;methodological defects in the current state of scientific practice&quot;. And there aren&#x27;t simple solutions &quot;to fix these defects&quot;.<p>(Having said that, the article is great. Many proposals (some of them mentioned in the article), such as a standard 0.5% significance level, more Bayes, awareness of power in addition to significance, avoiding p-hacking, etc., are important and useful steps. But they still won&#x27;t allow your opponent to prove that you were cheating :-)