Data Science, Statistics, Permutation Test, Monte Carlo Method.
also be normally distributed. 5. Draw histogram using ctr differences and see if true_ctr_diff lies within confidence interval. Since our distribution is normally distributed we can use empirical rule. 1.15 is within 95% confidence interval [-1.92, 1.96] therefore our mean CTR difference of AB test