Annoying A/B testing mistakes
I want an A/B test framework that automatically optimizes the size of the groups to maximize revenue.<p>At first, it would pick say a 50/50 split.   Then as data rolls in that shows group A is more likely to convert, shift more users over to group A.   Keep a few users on B to keep gathering data.    Eventually, when enough data has come in, it might turn out that flow A doesn't work at all for users in France - so the ideal would be for most users in France to end up in group B, whereas the rest of the world is in group A.<p>I want the framework to do all this behind the scenes - and preferably with statistical rigorousness.   And then to tell me which groups have diminished to near zero (allowing me to remove the associated code).