Meta-experiments: Improving experimentation through experimentation
<|reference_start|>Meta-experiments: Improving experimentation through experimentation: A/B testing is widexly used in the industry to optimize customer facing websites. Many companies employ experimentation specialists to facilitate and improve the process of A/B testing. Here, we present the application of A/B testing to this improvement effort itself, by running experiments on the experimentation process, which we call 'meta-experiments'. We discuss the challenges of this approach using the example of one of our meta-experiments, which helped experimenters to run more sufficiently powered A/B tests. We also point out the benefits of 'dog fooding' for the experimentation specialists when running their own experiments.<|reference_end|>
@article{müller2024meta-experiments:,
    title={Meta-experiments: Improving experimentation through experimentation},
    author={Melanie J.I. M"uller},
    journal={arXiv preprint arXiv:2406.16629},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.16629},
    primaryClass={cs.IR stat.AP}
}