Ab Testing, Experiment, Data Science, Machine Learning.
My experiment setup is a bit trivial since I picked a covariate that has a high correlation with the outcome. Because of that, we will see a lot of variance reduction. Specifically, I’m seeing the following in variance reduction and subject requirements for power of 0.8 (I’m using the average of