title: CobBO: Coordinate Backoff Bayesian Optimization with Two-Stage Kernels
abstract: Bayesian optimization is a popular method for optimizing expensive black-box functions. Yet it oftentimes struggles in high dimensions where a sufficient estimation of the global landscape requires more observations and the computation cost becomes prohibitively expensive.
  We introduce Coordinate backoff Bayesian optimization (CobBO) with two-stage kernels to alleviate this problem.
  In each iteration, a promising subset of coordinates is selected in the first stage, as past observed points in the full space are projected to the selected subspace adopting a simple kernel that sacrifices the approximation accuracy for computational efficiency.
  Then in the second stage of the same iteration a more sophisticated kernel is applied for estimating the landscape in the selected low dimensional subspace where the computational cost becomes affordable. Effectively, this second stage kernel refines the approximation of the global landscape estimated by the first stage kernel through a sequence of observations in the local subspace. This refinement lasts until a stopping rule is met determining when to back off from a certain subspace and switch to another coordinate subset.
  This decoupling significantly reduces the computational burden in high dimensions, while the two-stage kernels of the Gaussian process regressions fully leverage the observations in the whole space rather than only relying on observations in each coordinate subspace.
  Extensive evaluations show that CobBO finds solutions comparable to or better than other state-of-the-art methods for dimensions ranging from tens to hundreds, while reducing the trial complexity and computational costs.
authors: Jian Tan Niv Nayman MengChang WANG Rong Jin
hIndex: MengChang WANG 5