Online Policy Learning and Inference by Matrix Completion
<|reference_start|>Online Policy Learning and Inference by Matrix Completion: Making online decisions can be challenging when features are sparse and orthogonal to historical ones, especially when the optimal policy is learned through collaborative filtering. We formulate the problem as a matrix completion bandit (MCB), where the expected reward under each arm is characterized by an unknown low-rank matrix. The $\epsilon$-greedy bandit and the online gradient descent algorithm are explored. Policy learning and regret performance are studied under a specific schedule for exploration probabilities and step sizes. A faster decaying exploration probability yields smaller regret but learns the optimal policy less accurately. We investigate an online debiasing method based on inverse propensity weighting (IPW) and a general framework for online policy inference. The IPW-based estimators are asymptotically normal under mild arm-optimality conditions. Numerical simulations corroborate our theoretical findings. Our methods are applied to the San Francisco parking pricing project data, revealing intriguing discoveries and outperforming the benchmark policy.<|reference_end|>
@article{duan2024online,
    title={Online Policy Learning and Inference by Matrix Completion},
    author={Congyuan Duan, Jingyang Li and Dong Xia},
    journal={arXiv preprint arXiv:2404.17398},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.17398},
    primaryClass={stat.ML cs.LG}
}