Estimating Treatment Effects under Recommender Interference: A Structured Neural Networks Approach
<|reference_start|>Estimating Treatment Effects under Recommender Interference: A Structured Neural Networks Approach: Recommender systems are essential for content-sharing platforms by curating personalized content. To evaluate updates to recommender systems targeting content creators, platforms frequently rely on creator-side randomized experiments. The treatment effect measures the change in outcomes when a new algorithm is implemented compared to the status quo. We show that the standard difference-in-means estimator can lead to biased estimates due to recommender interference that arises when treated and control creators compete for exposure. We propose a "recommender choice model" that describes which item gets exposed from a pool containing both treated and control items. By combining a structural choice model with neural networks, this framework directly models the interference pathway while accounting for rich viewer-content heterogeneity. We construct a debiased estimator of the treatment effect and prove it is $\sqrt n$-consistent and asymptotically normal with potentially correlated samples. We validate our estimator's empirical performance with a field experiment on Weixin short-video platform. In addition to the standard creator-side experiment, we conduct a costly double-sided randomization design to obtain a benchmark estimate free from interference bias. We show that the proposed estimator yields results comparable to the benchmark, whereas the standard difference-in-means estimator can exhibit significant bias and even produce reversed signs.<|reference_end|>
@article{zhan2024estimating,
    title={Estimating Treatment Effects under Recommender Interference: A
  Structured Neural Networks Approach},
    author={Ruohan Zhan, Shichao Han, Yuchen Hu, Zhenling Jiang},
    journal={arXiv preprint arXiv:2406.14380},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.14380},
    primaryClass={econ.EM cs.LG stat.ME}
}