Maximizing acquisition functions for Bayesian optimization
Bayesian optimization is a sample-efficient approach to global optimization that relies on theoretically motivated value heuristics (acquisition functions) to guide its search process. Fully maximizing acquisition functions produces the Bayes' decision rule, but this ideal is difficult to achieve since these functions are frequently non-trivial to optimize. This statement is especially true when evaluating queries in parallel, where acquisition functions are routinely non-convex, highdimensional, and intractable. We first show that acquisition functions estimated via Monte Carlo integration are consistently amenable to gradient-based optimization. Subsequently, we identify a common family of acquisition functions, including EI and UCB, whose properties not only facilitate but justify use of greedy approaches for their maximization.
Summary: This paper proposes a new way of solving the problem of optimizing the acquisition function in Bayesian optimization. The idea is to look at most common acquisitions as the optimization of an expectation of a function, which can be reformulated using the 'reparametrization trick' and optimized using Monte Carlo. The papers shows the reformulation of various acquisitions and presents some interesting experimental results.  Strengths and weaknesses of the submission: The paper is really well written and it attacks a problem that affects seriously the performance of new Bayesian optimization methods. The approach is very original and well illustrated and I think that this approach has the potential to become a standard practice in Bayesian optimization alternatively to current approaches that are not even well formalized (each BayesOpt library does it it in its own way). In my opinion this paper brings some foundational aspects to a part of the the optimization loop that has been ignored by the community, in general.  Quality: this is a high quality work both because the idea is new but also because the paper is very nicely written.  Clarity: the paper is very clear   Originality: The method to reparametrize the acquisition is know but its application to this domain is new.  Significance: the method is significant for the BayesOpt community in particular and for the ML community in general (given that these techniques are becoming part of the standard toolkit of the ML practitioner).