A much simpler approach is to AABB test instead of AB test. Rather than splitting your users into 2 buckets (A and B), split them into 4 buckets (A1, A2, B1, B2). Give groups A1 and A2 one variation and groups B1 and B2 the other variation. When A1 equals A2 and B1 equals B2 then you know you have statistical significance and you can compare A1+A2 to B1+B2.