Data Science, Variance, A B Testing, LinkedIn.
prediction error: Figure 5: formula for the adjusted variances of our north star using a surrogate metric. Image by author. This adjusted variance comes in to play when when calculating statistical significance. Below, we use a the corrected variance in formula for a t-statistic: Figure 6: formula