Using loaded dice to cheat at Settlers of Catan, and p-values to avoid suspicion
Very nice hack and analysis thereof.I do take issue with this statement, though, in particular the part highlighted (by me):> "it’s impossible for your opponents to scientifically prove that you’re cheating. This impossibility is due to methodological defects in the current state of scientific practice, and we’ll highlight some ongoing work to fix these defects."The impossibility is not due to methodological defects in the current state of scientific practice. It's intrinsic to the world. If you cheat only a bit, and only rarely, it might be impossible to detect it statistically.Of course, there are problems with the current state of scientific practice and the mindless application of statistics (particularly the 5% significance level). The replicability crisis in psychology comes to mind.However, I think statistics is unfairly maligned here. Statistics mostly delivers on its promise, it just promises less than what some people think.The world is messy, and statistics can help us to make sense of it, but it can only do so much. In particular, it's not magic. You need many observations to make statistical statements, and more observations to make more precise statements (or statements at a higher significance level). That's intrinsic to how the world is. It's not a problem with "methodological defects in the current state of scientific practice". And there aren't simple solutions "to fix these defects".(Having said that, the article is great. Many proposals (some of them mentioned in the article), such as a standard 0.5% significance level, more Bayes, awareness of power in addition to significance, avoiding p-hacking, etc., are important and useful steps. But they still won't allow your opponent to prove that you were cheating :-)