Epsilon-Greedy Thompson Sampling to Bayesian Optimization
<|reference_start|>Epsilon-Greedy Thompson Sampling to Bayesian Optimization: Bayesian optimization (BO) has become a powerful tool for solving simulation-based engineering optimization problems thanks to its ability to integrate physical and mathematical understandings, consider uncertainty, and address the exploitation--exploration dilemma. Thompson sampling (TS) is a preferred solution for BO to handle the exploitation--exploration trade-off. While it prioritizes exploration by generating and minimizing random sample paths from probabilistic models -- a fundamental ingredient of BO -- TS weakly manages exploitation by gathering information about the true objective function after it obtains new observations. In this work, we improve the exploitation of TS by incorporating the $\varepsilon$-greedy policy, a well-established selection strategy in reinforcement learning. We first delineate two extremes of TS, namely the generic TS and the sample-average TS. The former promotes exploration, while the latter favors exploitation. We then adopt the $\varepsilon$-greedy policy to randomly switch between these two extremes. Small and large values of $\varepsilon$ govern exploitation and exploration, respectively. By minimizing two benchmark functions and solving an inverse problem of a steel cantilever beam,we empirically show that $\varepsilon$-greedy TS equipped with an appropriate $\varepsilon$ is more robust than its two extremes,matching or outperforming the better of the generic TS and the sample-average TS.<|reference_end|>
@article{do2024epsilon-greedy,
    title={Epsilon-Greedy Thompson Sampling to Bayesian Optimization},
    author={Bach Do and Taiwo Adebiyi and Ruda Zhang},
    journal={arXiv preprint arXiv:2403.00540},
    year={2024},
    doi={10.1115/1.4066858},
    archivePrefix={arXiv},
    eprint={2403.00540},
    primaryClass={cs.LG math.OC stat.ML}
}