Bayesian Optimization with Gradients
Bayesian optimization has been successful at global optimization of expensiveto-evaluate multimodal objective functions. However, unlike most optimization methods, Bayesian optimization typically does not use derivative information. In this paper we show how Bayesian optimization can exploit derivative information to find good solutions with fewer objective function evaluations. In particular, we develop a novel Bayesian optimization algorithm, the derivative-enabled knowledgegradient (d-KG), which is one-step Bayes-optimal, asymptotically consistent, and provides greater one-step value of information than in the derivative-free setting. d-KG accommodates noisy and incomplete derivative information, comes in both sequential and batch forms, and can optionally reduce the computational cost of inference through automatically selected retention of a single directional derivative. We also compute the d-KG acquisition function and its gradient using a novel fast discretization-free technique. We show d-KG provides state-of-the-art performance compared to a wide range of optimization procedures with and without gradients, on benchmarks including logistic regression, deep learning, kernel learning, and k-nearest neighbors.
Major: 
To me this paper is an important one as it presents substantial contributions to Bayesian Optimization that are at the same time elegant and practically efficient. While the paper is well written, it could be formally improved in places; below are 
a few comments questions that might be useful toward such improvements. Besides this, I was a bit puzzled by the fact that Theorem 1 is not very precisely stated; also, given the fact that it is proven in appendix (which is understandable given 
the space limitations) and with a proof that requires substantial time to proofread formally (as it is almost a second paper in itself, with a specific policy formalism/set-up), I had to adapt my confidence score accordingly. Also, I was wondering if the consistency proof could work in the case of noiseless observations, where it does not make sense to replicate evaluations at the same point(s)?
Nevertheless, from the methodological contributions and their practical relevance, I am conviced that the presented approach should be spread to BO/ML researchers and that NIPS is an excellent platform to do so.   
Minor:
* In the abstract: "most optimization methods" is a bit strong. Derivative-free optimization methods should not be forgotten. 
Yet, gradients are indeed essential in (differentiable) local optimization methods. 
* In the abstract: syntax issue in "We show d-KG provides". 
* Introduction: from "By contrast", see comment above regarding *local* optimization.
* Introduction: "relative value" does not sound very clear to me. Maybe speak of "variations" or so? 
* Introduction: "we explore the what, when, and why". In a way yes, but the main emphasis is on d-KG, and even if generalities 
and other approaches are tackled, it is probably a bit strong to claim/imply that *Bayesian Optimization with Gradients* is 
*explored* in a paper subjected to such space restrictions. 
* Introduction: from "And even" to "finite differences"; in such case, why not use this credit to do more criterion-driven 
evaluations? Are there good reasons to believe that two neigbouring points approximating a derivative will be better than 
two (more) distant well-chosen points? [NB: could there exist a connection to "twin points", in Gaussian kernel settings?] 
* Section 3.1: "to find argmin"; so, the set [in case of several minimizers]?
* Section 3.1: "linear operator" calls for additional regularity condition(s). By the way, it would be nice to ensure 
differentiability of the GP via the chosen mu and K. 
* Section 3.1: the notation used in equation (3.2) seem to imply that there cannot be multiple evaluations at the same x. 
Notations in terms of y_i (and similar for the gradient), corresponding to points x_i, could accomodate that. 
* Section 3.1: the paragraph from line 121 to 126 sounds a bit confusing to me (I found the writing rather cryptic there)
* Section 3.2: "expected loss"; one would expect a translation by the actual min there (so that loss=0 for successful optim)
* Section 3.2: in equation (3.3), conditioning on "z" is abusive. By the way, why this "z" notation (and z^(i)) for "x" points? 
* Algorithm 2, lines 2: a) =argmax abusive and b) is theta on the unit sphere? restricted to canonical directions? 
* Section 3.3: in the expression of the d-KG factor, the term \hat{\sigma}^{(n)} appears to be related to (a generalization of) 
kriging update formulas for batch-sequential data assimilation. Maybe it could make sense to connect one result ro the other?
* Section 3.3: the approach with the enveloppe theorem is really nice. However, how are the x^*(W) estimated? Detail needed...
* Section 3.3: "K samples"; from which distribution? + Be careful, the letter K was already used before, for the covariance. 
* Section 3.3, line 191: "conditioning" does not seem right here...conditional? conditionally? 
* Section 3.4: in Prop.1, you mean for all z^{(1:q)}? Globally the propositions lack mathematical precision...
* Section 3.4: ...in particular, what are the assumptions in Theorem 1? E.g., Is f assumed to be drawm from the running GP? 
* Section 4: notation Sigma is used for the covariance kernel while K was used before. 
* Section 4: about "the number of function evaluations"; do the figures account for the fact that "d-approaches" do several f 
evaluations at each iteration?
* Section 5: about "low dimensional"; not necessarily...See REMBO and also recent attempts to combine BO and kernel (sparse) decompositions.