The Parallel Knowledge Gradient Method for Batch Bayesian Optimization
In many applications of black-box optimization, one can evaluate multiple points simultaneously, e.g. when evaluating the performances of several different neural networks in a parallel computing environment. In this paper, we develop a novel batch Bayesian optimization algorithm -the parallel knowledge gradient method. By construction, this method provides the one-step Bayes optimal batch of points to sample. We provide an efficient strategy for computing this Bayes-optimal batch of points, and we demonstrate that the parallel knowledge gradient method finds global optima significantly faster than previous batch Bayesian optimization algorithms on both synthetic test functions and when tuning hyperparameters of practical machine learning algorithms, especially when function evaluations are noisy.
This work provides a method to perform parallel Batch Bayesian Optimization by computing a Bayes-optimal batch of configurations to evaluate next. The proposed method measures the utility of evaluating a batch of configurations using the knowledge gradient (q-KG). To efficiently maximize the acquisition function they present a method based on infinitesimal perturbation analysis to estimate the gradient of q-KG. In 3 experiments they show, that q-KG performs competitive with existing parallel Bayesian optimization methods on problems without observation noise and it outperforms these methods on problems with observation noise.
I do like the idea of performing parallel hyperparameter optimization to allow the efficient use of multi-core environments, but some aspects remain unclear to me and I am not convinced by the experimental results. Please find feedback for each of the points above and some questions at the end:  # Technical quality The paper used common artificial benchmark function to show superiority of the proposed method. Additional to these cheap-to-evaluate problems the paper also presents results for optimizing a CNN and logistic regression for Cifar10/MNIST. In the experiment section I would like to see an experiment showing wall-clock time or an analysis on how much one can gain from using this parallel method compared to simple random search or sequential optimization. Also I do not see, why q should be set to 4 for all experiments.  # Novelty/Originality The paper proposes to use the knowledge gradient (a generalization of expected improvement) as an acquisition function and infinitesimal perturbation analysis (IPA) to maximize q-KG. Both methods themselves are not new and the paper refers to "Parallel Bayesian global optimization of expensive functions", which already shows how to use IPA to select a batch of configurations using expected improvement.  # Impact/Usefulness Parallel hyperparameter tunings method are highly relevant for (real-world) applications from many disciplines, such as machine learning, computer vision, biology and robotics.  # Clarity/Presentation The technical details and the background section are described well and the experiment section contains everything necessary to comprehend the performed comparisons.  Questions to the authors:   1.) As Wang et al. (2015) seems to be very relevant to this work and code seems to be available online, have you compared to this method? (Or is this "parallel EI in MOE", then you should call it q-EI to distinguish it from Spearmint's parallel method). To avoid confusion it would be beneficial to clearly state the differences in the paper.   2.) Nowadays multi-core systems allow for much more than 4 parallel runs. For example     * "Scalable Bayesian Optimization Using Deep Neural Networks", ICML'15 used up to 800 parallel runs of DNGO     * "Parallel Algorithm Configuration", LION'12 used 25 SMAC runs in parallel      * "Practical Bayesian Optimization of Machine Learning Algorithms", NIPS'12 used up to 10 parallel runs of Spearmint      Do you have any insights to what extent your method scales and how much one can gain from more parallel runs?   3.) Right now your method is supposed to run synchronously. Would it be possible to extend it to run asynchronously, such as for example Spearmint does?   4.) In the end, wall-clock time matters. Have you evaluated how your method compares to random search or simple sequential Bayesian optimization with respect to wall-clock time? How large is the overhead for computing a new batch?    5.) What does "iterations" as the x-label in your plots mean. Is it the number of function evaluations or number of batches? And does the x-axis include the initial design? If yes, why do not all methods start from the same value?  Minor comments:  * Is there a z^(1:n) missing in eq (4.1)?  * "this architecture has been carefully tuned by the human expert, and achieve a test error" -> achieves  * Parenthetical references are not nouns, e.g. "[16] suggests parallelizing" should be "Snoek et al. [16] suggest parallelizing".  * I don't think "Wang et al. (2015)" is the correct reference for IPA, as they do not introduce the method, but use and apply it.