Principled Preferential Bayesian Optimization
<|reference_start|>Principled Preferential Bayesian Optimization: We study the problem of preferential Bayesian optimization (BO), where we aim to optimize a black-box function with only preference feedback over a pair of candidate solutions. Inspired by the likelihood ratio idea, we construct a confidence set of the black-box function using only the preference feedback. An optimistic algorithm with an efficient computational method is then developed to solve the problem, which enjoys an information-theoretic bound on the total cumulative regret, a first-of-its-kind for preferential BO. This bound further allows us to design a scheme to report an estimated best solution, with a guaranteed convergence rate. Experimental results on sampled instances from Gaussian processes, standard test functions, and a thermal comfort optimization problem all show that our method stably achieves better or competitive performance as compared to the existing state-of-the-art heuristics, which, however, do not have theoretical guarantees on regret bounds or convergence.<|reference_end|>
@article{xu2024principled,
    title={Principled Preferential Bayesian Optimization},
    author={Wenjie Xu, Wenbin Wang, Yuning Jiang, Bratislav Svetozarevic, Colin N.
  Jones},
    journal={arXiv preprint arXiv:2402.05367},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.05367},
    primaryClass={cs.LG}
}