Adversarially Robust Optimization with Gaussian Processes
In this paper, we consider the problem of Gaussian process (GP) optimization with an added robustness requirement: The returned point may be perturbed by an adversary, and we require the function value to remain as high as possible even after this perturbation. This problem is motivated by settings in which the underlying functions during optimization and implementation stages are different, or when one is interested in finding an entire region of good inputs rather than only a single point. We show that standard GP optimization algorithms do not exhibit the desired robustness properties, and provide a novel confidence-bound based algorithm STABLEOPT for this purpose. We rigorously establish the required number of samples for STABLEOPT to find a near-optimal point, and we complement this guarantee with an algorithm-independent lower bound. We experimentally demonstrate several potential applications of interest using real-world data sets, and we show that STABLEOPT consistently succeeds in finding a stable maximizer where several baseline methods fail.
The authors consider a problem of black-box optimisation with additional robustness requirements. As opposed to standard Bayesian optimisation, where the unknown function f is modelled typically with GP and new evaluation location chosen to improve the knowledge of the global optimiser sequentially, the authors instead want to find a point whose objective function value remains almost optimal even after some perturbation of the point. In this setting e.g. a global optimiser that lies on top of a narrow peak might be suboptimal choice because even a small change of the point could change its value drastically while a point having smaller objective function value but lying in a more flat region might be better choice.  The authors formulate the problem so that instead of trying to find a global optimiser, they look for a point that is the maximiser of the minimum of f over some \epsilon-neighbourhood of the point. They extend the well-known GP-UCB/LCB strategy to this robust setting and provide both theoretical and empirical support for their method. Also, they show that many "max-min" type of optimisation problems in literature can be seen as special cases of their approach.  This is a very strong and well-written paper on an interesting problem and, as far as I know, the topic has not been studied in such detail previously. The authors also list some practical applications for their method. While I consider myself as an expert in BayesOpt and related methods and could understand the paper quite well, I am not familiar with the underlying theoretical framework (based on e.g. [30]) and could not check the proofs of the theoretical results. However, the theoretical results also look carefully done.   I have the following minor concerns and comments/questions of the paper:  - I found no discussion on how the optimisation of (10) is carried out and what is the computational cost as compared to standard BayesOpt methods like GP-UCB or the time required to evaluate f.  Optimisation of (10) may not be straightforward because the nested minimisation over the \epsilon ball. Perhaps these computational aspects could be briefly discussed in the supplement. Experiments focus on rather low-dimensional cases where grid optimisation might already work just fine but I also wonder how would the approach work in high-dimensions?  - There appears to be large difference between GP-UCB and STABLE-GP-UCB in Fig2c and especially in Fig7c although the only difference between these two appears to be the selection of the reported point. I thus wonder if MAXIMIN-GP-UCB would work better if the reported point was chosen similarly as with STABLE-* methods.   - lines 118-220: Generally, I am a bit puzzled on how the reported point should be chosen in the first place. Maybe I am missing something here but why not just use the GP mean function as a proxy to the unknown function f and then use (5) to select the final reported point? Or is it author's experience that the (pessimistic) estimate based on the lcb criteria actually works better in practice?  - line 225: Comparison to only GP-UCB-based methods seems reasonable but I believe that one could also extend EI and entropy search to this robust setting (evaluating the resulting acquisition function might require numerical computations though).  - line 221: Choosing \beta_t^{1/2}=2 seems a bit arbitrary choice and I wonder what is actually going on here. I could imagine that a more conservative strategy that produces more exploration of the parameter space might actually be useful here.  - Distance d(x,x') is also an input of Algorithm 1, right?  - I wonder what GP model was used in Robust robot pushing experiment with the two functions f_i, i=1,2?  - For consistency, the order of the methods in the legend box of Fig3 could be the same as in other Figs.   Edit after author rebuttal: After reading the rebuttal and other reviews I remain confident that this is high quality work and should be accepted. However, I am still a bit puzzled about some design choices and how they would affect the experimental results in practice, so I hope the authors can indeed clarify these in the final version.