A lot of math detail, but a naive reader reading it will still make the mistake outlined in <a href="http:&#x2F;&#x2F;www.evanmiller.org&#x2F;how-not-to-run-an-ab-test.html" rel="nofollow">http:&#x2F;&#x2F;www.evanmiller.org&#x2F;how-not-to-run-an-ab-test.html</a>.  And a less naive reader will still be left with no guidance on how to make real-world decisions if you weren&#x27;t so lucky as to get a strong statistical result.<p>Here is a much, much simpler strategy that I can suggest for business users.<p>1. Figure out the longest that you can afford to run your test for.<p>2. Figure out the number of conversions N you expect in that time.<p>3. Start running your test, randomly splitting visitors into A and B.<p>4. Stop the test as soon as one version has sqrt(N) more conversions than the other.  Else wait until you get to N conversions between them, and go with what is ahead.<p>Here are some comments on the procedure.<p>Stopping before N&#x2F;2 total conversions is roughly the same certainty as stopping at 95% confidence.  Stopping after that is an admission that you are crossing your fingers and going with an educated guess, and it is not feasible to collect enough data to get a better answer.<p>If one version has a conversion rate which is (1 + 2&#x2F;sqrt(n)) better than the other, you pretty reliably choose right.  The flip side of this is that you&#x27;ll make a lot of mistakes if you get much below that threshold.  If those potential errors are too big, then A&#x2F;B testing is not going to work well for you because you don&#x27;t have enough data for statistics.