Understanding the Effects of the Baidu-ULTR Logging Policy on Two-Tower Models
<|reference_start|>Understanding the Effects of the Baidu-ULTR Logging Policy on Two-Tower Models: Despite the popularity of the two-tower model for unbiased learning to rank (ULTR) tasks, recent work suggests that it suffers from a major limitation that could lead to its collapse in industry applications: the problem of logging policy confounding. Several potential solutions have even been proposed; however, the evaluation of these methods was mostly conducted using semi-synthetic simulation experiments. This paper bridges the gap between theory and practice by investigating the confounding problem on the largest real-world dataset, Baidu-ULTR. Our main contributions are threefold: 1) we show that the conditions for the confounding problem are given on Baidu-ULTR, 2) the confounding problem bears no significant effect on the two-tower model, and 3) we point to a potential mismatch between expert annotations, the golden standard in ULTR, and user click behavior.<|reference_end|>
@article{de haan2024understanding,
    title={Understanding the Effects of the Baidu-ULTR Logging Policy on Two-Tower
  Models},
    author={Morris de Haan, Philipp Hager},
    journal={arXiv preprint arXiv:2409.12043},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.12043},
    primaryClass={cs.IR cs.LG}
}