  Efficient Global Optimization (EGO) is the canonical form of Bayesian
optimization that has been successfully applied to solve global optimization of
expensive-to-evaluate black-box problems. However, EGO struggles to scale with
dimension, and offers limited theoretical guarantees. In this work, a
trust-region framework for EGO (TREGO) is proposed and analyzed. TREGO
alternates between regular EGO steps and local steps within a trust region. By
following a classical scheme for the trust region (based on a sufficient
decrease condition), the proposed algorithm enjoys global convergence
properties, while departing from EGO only for a subset of optimization steps.
Using extensive numerical experiments based on the well-known COCO {bound
constrained problems}, we first analyze the sensitivity of TREGO to its own
parameters, then show that the resulting algorithm is consistently
outperforming EGO and getting competitive with other state-of-the-art black-box
optimization methods.

Optimization and Control;Machine Learning