&gt; Statistical power is simply the likelihood that the difference you’ve detected during your experiment actually reflects a difference in the real world.<p>This seems incorrect to me. Isn&#x27;t statistical power the likelihood that the null hypothesis would generate an outcome at least as extreme as what you observed?<p>I&#x27;m guessing the issue has a lot more to do with peeking at the outcome and not correcting for it (and similarly running many tests)<p><a href="http:&#x2F;&#x2F;www.stat.columbia.edu&#x2F;~gelman&#x2F;research&#x2F;unpublished&#x2F;p_hacking.pdf" rel="nofollow">http:&#x2F;&#x2F;www.stat.columbia.edu&#x2F;~gelman&#x2F;research&#x2F;unpublished&#x2F;p_...</a>