Towards Efficient and Optimal Covariance-Adaptive Algorithms for Combinatorial Semi-Bandits
<|reference_start|>Towards Efficient and Optimal Covariance-Adaptive Algorithms for Combinatorial Semi-Bandits: We address the problem of stochastic combinatorial semi-bandits, where a player selects among $P$ actions from the power set of a set containing $d$ base items. Adaptivity to the problem's structure is essential in order to obtain optimal regret upper bounds. As estimating the coefficients of a covariance matrix can be manageable in practice, leveraging them should improve the regret. We design ``optimistic'' covariance-adaptive algorithms relying on online estimations of the covariance structure, called OLSUCBC and COSV (only the variances for the latter). They both yields improved gap-free regret. Although COSV can be slightly suboptimal, it improves on computational complexity by taking inspiration from Thompson Sampling approaches. It is the first sampling-based algorithm satisfying a $\sqrt{T}$ gap-free regret (up to poly-logs). We also show that in some cases, our approach efficiently leverages the semi-bandit feedback and outperforms bandit feedback approaches, not only in exponential regimes where $P\gg d$ but also when $P\leq d$, which is not covered by existing analyses.<|reference_end|>
@article{zhou2024towards,
    title={Towards Efficient and Optimal Covariance-Adaptive Algorithms for
  Combinatorial Semi-Bandits},
    author={Julien Zhou (Criteo, Inria), Pierre Gaillard (Inria), Thibaud Rahier
  (Criteo), Houssam Zenati (Inria), Julyan Arbel (Inria)},
    journal={arXiv preprint arXiv:2402.15171},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.15171},
    primaryClass={cs.LG math.ST stat.ML stat.TH}
}