Regret Analysis for Randomized Gaussian Process Upper Confidence Bound
<|reference_start|>Regret Analysis for Randomized Gaussian Process Upper Confidence Bound: Gaussian process upper confidence bound (GP-UCB) is a theoretically established algorithm for Bayesian optimization (BO), where we assume the objective function $f$ follows GP. One notable drawback of GP-UCB is that the theoretical confidence parameter $\beta$ increased along with the iterations is too large. To alleviate this drawback, this paper analyzes the randomized variant of GP-UCB called improved randomized GP-UCB (IRGP-UCB), which uses the confidence parameter generated from the shifted exponential distribution. We analyze the expected regret and conditional expected regret, where the expectation and the probability are taken respectively with $f$ and noises and with the randomness of the BO algorithm. In both regret analyses, IRGP-UCB achieves a sub-linear regret upper bound without increasing the confidence parameter if the input domain is finite. Finally, we show numerical experiments using synthetic and benchmark functions and real-world emulators.<|reference_end|>
@article{takeno2024regret,
    title={Regret Analysis for Randomized Gaussian Process Upper Confidence Bound},
    author={Shion Takeno, Yu Inatsu, Masayuki Karasuyama},
    journal={arXiv preprint arXiv:2409.00979},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.00979},
    primaryClass={cs.LG stat.ML}
}