Leveraging (Biased) Information: Multi-armed Bandits with Offline Data
<|reference_start|>Leveraging (Biased) Information: Multi-armed Bandits with Offline Data: We leverage offline data to facilitate online learning in stochastic multi-armed bandits. The probability distributions that govern the offline data and the online rewards can be different. Without any non-trivial upper bound on their difference, we show that no non-anticipatory policy can outperform the UCB policy by (Auer et al. 2002), even in the presence of offline data. In complement, we propose an online policy MIN-UCB, which outperforms UCB when a non-trivial upper bound is given. MIN-UCB adaptively chooses to utilize the offline data when they are deemed informative, and to ignore them otherwise. MIN-UCB is shown to be tight in terms of both instance independent and dependent regret bounds. Finally, we corroborate the theoretical results with numerical experiments.<|reference_end|>
@article{cheung2024leveraging,
    title={Leveraging (Biased) Information: Multi-armed Bandits with Offline Data},
    author={Wang Chi Cheung, Lixing Lyu},
    journal={arXiv preprint arXiv:2405.02594},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.02594},
    primaryClass={cs.LG stat.ML}
}