High-Dimensional Gaussian Process Bandits
Many applications in machine learning require optimizing unknown functions defined over a high-dimensional space from noisy samples that are expensive to obtain. We address this notoriously hard challenge, under the assumptions that the function varies only along some low-dimensional subspace and is smooth (i.e., it has a low norm in a Reproducible Kernel Hilbert Space). In particular, we present the SI-BO algorithm, which leverages recent low-rank matrix recovery techniques to learn the underlying subspace of the unknown function and applies Gaussian Process Upper Confidence sampling for optimization of the function. We carefully calibrate the exploration-exploitation tradeoff by allocating the sampling budget to subspace estimation and function optimization, and obtain the first subexponential cumulative regret bounds and convergence rates for Bayesian optimization in high-dimensions under noisy observations. Numerical results demonstrate the effectiveness of our approach in difficult scenarios.
This paper considers optimization of high-dimensional 
      expensive-to-compute non-convex objective functions, under the 
      assumption that the high-dimensional function only varies in a 
      low-dimensional subspace. The paper proposes an algorithm with two 
      steps. The first step is to estimate the low-dimensional subspace, and 
      the second step is to apply the GP-UCB algorithm to the estimated 
      low-dimensional objective function. Overall, I think the paper 
      is interesting, and applies a reasonable idea to an important and 
      difficult problem. I did however, see a number of practical and 
      algorithmic drawbacks to the proposed approach, some potential issues 
      with the theoretical analysis, and some weak points in the numerical 
      experiments. These are described in more detail below. 1. 
      Although the method is interesting, there are several drawbacks in its 
      design: a. The method does not improve its initial estimate of the 
      subspace while in the second optimization stage. b. It requires 
      knowing several constants which would be hard to know in practice. k, 
      the number of dimensions in the low-dimensional subspace B, an upper 
      bound on the RKHS norm of the function being optimize sigma, the 
      variance of the noise c. It requires making some assumptions about the 
      underlying function g (in C^2 with Lipschitz continuous second order 
      derivatives, and a full rank Hession at 0). It is claimed that these 
      assumptions are "satisfied for many functions", which is true, but in 
      practice, for a given high-dimensional non-convex expensive-to-compute 
      function to which we want to apply this algorithm, I think it is 
      unlikely that we would know whether or not that function satisfied these 
      assumptions. It also requires assuming that the noise is Gaussian with 
      constant variance. Though these assumptions may be unavoidable, it 
      would be better to do numerical experiments to investigate whether 
      failing to meet the assumptions causes the behavior of the proposed 
      algorithm to decay significantly. d. Since \hat{\sigma} depends on 
      T (see point below) and GP-UCB is run with parameter \hat{\sigma}, we 
      must decide in practice how many iterations we will run this algorithm 
      before we start running it. If we decide later that we want to run 
      more samples later, we have to restart from scratch or lose our 
      theoretical guarantee. e. The proposed algorithm does not do very 
      well on simple regret, as the very simple algorithm RandomH-UCB 
      outperforms it on two out of three problem settings. f. There is no 
      way to include prior information from the practitioner about which 
      subspaces are likely to be important. In my practical experience with 
      these kinds of problems, it is common that the person close to the 
      application has a good idea of which coordinate directions are likely 
      to matter, and which are not. It is only on the margins where they are 
      uncertain. (It is true that it is more difficult to ask a practitioner 
      which subspaces, not necessarily aligned with coordinate axes, are 
      likely to matter). Indeed, a good benchmark algorithm for comparison 
      would be to take a real problem, ask a person close to the problem 
      which 5 coordinate directions he thinks are most important, and run 
      GP-UCB on just those coordinate directions, picking arbitrary values 
      for the other coordinates. g. The assumption is made that there is 
      absolutely no variation in the function outside of the low-dimensional 
      subspace. In practice, I think this is unlikely to be true. Instead, 
      it seems more likely that there is a small amount of variation outside 
      of the low-dimensional subspace. 2. In the choice for \hat{\sigma} 
      on line 213, T does not appear. Of course, it should appear, since the 
      probability that the maximum of T iid normal(0,sigma) random variables 
      exceeds any fixed threshold goes to 0 as T goes to infinity. My 
      understanding from reading the surrounding text is that the omission 
      of T is just a typo, but there appears to be an error in the analysis 
      later resulting from this omission (see point 3b). 3. Two 
      potential issues with the proofs: a. B is specified as an upper bound 
      on the squared RKHS norm of the function \hat{g}. But in the case with 
      noisy measurements, \hat{g} is determined in part based on the 
      estimated subspace, which is random. So even though g is a 
      deterministic function, \hat{g} is random, and so its squared RKHS 
      norm is random. How do we know there exists an upper bound B that 
      holds almost surely? b. \hat{\sigma} seems to depend on T, although an 
      apparent typo hides this dependence (see discussion in other comment 
      about line 213). Buried in the constant in Theorem 3 of the GP-UCB 
      paper [Srinivas et al. 2012] (upon which the proof of Theorem 1 is 
      based) is a dependence on \hat{\sigma}, which that paper assumes to be 
      a constant. This extra dependence on T should be included in the bound 
      on regret. 4. Theorem 5 uses g rather than \hat{g} in the term 
      R_{UCB}(T,g,\kappa), unlike Lemma 1, which used \hat{g}. I believe 
      this is a typo. If it is not a typo, how is moving from \hat{g} to g 
      accomplished? The proof doesn't make this clear. If it is simply a 
      typo, this bound is not as explicit as we would like, because 
      R_{UCB}(T,\hat{g},\kappa) is a random variable that depends in a 
      complicated way on the algorithm, rather than depending only on 
      constants. 5. I do not see precise guidance as to how to 
      choose the number of times we resample each point in practice, either 
      in Theorem 5 or in the section of Noisy Observations right before it. 
      Only its big-O dependence on other quantities is given. 6. 
      The numerical experiments are not especially thorough. a. They only 
      perform 20 replications which leaves the standard error quite wide. 
      Indeed, is the halfwidth of the bars just a single standard error? If 
      so, it looks like the standard errors are large enough that we can't 
      claim with statistical significance that SI-BO is better than either 
      RandomS-UCB or RandomH-UCB in most cases. Or am I misinterpreting the 
      figure? b. The paper does not compare with the method from reference 
      [19] in the paper, Wang et al. 2013. c. Only three test problems 
      are investigated, (all of which are just benchmark problems, and not 
      especially interesting). d. The values of k chosen are all really 
      small, only 1, 2 or 3. I would think that people would typically be 
      interested in running this kind of algorithm with slightly larger 
      values of k. Minor issues: Pertaining to the paragraph 
      in which line 213 appears, it is confusing that here T is the number 
      of GP-UCB iterations, but later, in Lemma 1, T is the overall number 
      of iterations. The figure labels (a) and (b) in the supplement 
      section C are reversed. In the supplement, Lemma 1 is called 
      Theorem 1.