Large-Scale Metric Computation in Online Controlled Experiment Platform
<|reference_start|>Large-Scale Metric Computation in Online Controlled Experiment Platform: Online controlled experiment (also called A/B test or experiment) is the most important tool for decision-making at a wide range of data-driven companies like Microsoft, Google, Meta, etc. Metric computation is the core procedure for reaching a conclusion during an experiment. With the growth of experiments and metrics in an experiment platform, computing metrics efficiently at scale becomes a non-trivial challenge. This work shows how metric computation in WeChat experiment platform can be done efficiently using bit-sliced index (BSI) arithmetic. This approach has been implemented in a real world system and the performance results are presented, showing that the BSI arithmetic approach is very suitable for large-scale metric computation scenarios.<|reference_end|>
@article{xiong2024large-scale,
    title={Large-Scale Metric Computation in Online Controlled Experiment Platform},
    author={Tao Xiong, Yong Wang},
    journal={PVLDB, 17(12): 4014 - 4024, 2024},
    year={2024},
    doi={10.14778/3685800.3685823},
    archivePrefix={arXiv},
    eprint={2405.08411},
    primaryClass={cs.DC}
}