Bayesian Optimization with Robust Bayesian Neural Networks
Bayesian optimization is a prominent method for optimizing expensive-to-evaluate black-box functions that is widely applied to tuning the hyperparameters of machine learning algorithms. Despite its successes, the prototypical Bayesian optimization approach -using Gaussian process models -does not scale well to either many hyperparameters or many function evaluations. Attacking this lack of scalability and flexibility is thus one of the key challenges of the field. We present a general approach for using flexible parametric models (neural networks) for Bayesian optimization, staying as close to a truly Bayesian treatment as possible. We obtain scalability through stochastic gradient Hamiltonian Monte Carlo, whose robustness we improve via a scale adaptation. Experiments including multi-task Bayesian optimization with 21 tasks, parallel optimization of deep neural networks and deep reinforcement learning show the power and flexibility of this approach.
This paper develops a novel approach to Bayesian optimization using Bayesian deep neural networks.  In particular, the authors combine recent advances in approximating Bayesian neural networks (stochastic gradient MCMC and Hamiltonian Monte Carlo) to be effective at Bayesian optimization.  Bayesian optimization is traditionally performed using Gaussian processes, but this does not scale well and thus becomes computationally intractable for many observations (or few observations across many different tasks).  The authors show that their approach usually outperforms the one existing formulation for Bayesian optimization with neural networks on a variety of optimization tasks.
It is very difficult to get most of the existing approaches to approximate Bayesian neural networks to work robustly without a significant amount of problem specific tuning (which makes them a poor choice for a task such as Bayesian optimization).  I really appreciate the authors' commitment to making the methodology generalize across different problems.  In general, I think this is an excellent paper.  One glaring omission in this paper is any evaluation of the wall-clock runtime of the approach.  How long does it really take in practice to perform 2000 burn-in steps and then 50*50 = 2500 steps for thinning and sampling?  I'm curious how many data points one could reasonably accommodate while performing a parallel Bayesian optimization before the optimization procedure becomes the bottleneck.  Having a latent vector representation for each experiment in the multitask setting seems quite sensible.  However, I don't see much detail about how these latent vectors are learned.  Are they learned in the same way as the model weights? What are the priors on these vectors?  The choice of prior could certainly have a non-trivial impact on the performance in Bayesian optimization.  e.g. setting the the prior to zero-mean Gaussian encodes a notion that all tasks are similar (i.e. all the zero-vector) in the absence of observations to the contrary.  That seems like a sensible prior for this, but it would be nice to have a discussion about this.  I disagree that a one-hot encoding would be less efficient; in fact I think it would be exactly the same.  With a one hot coding you are essentially just treating the weights attached to each input as the latent vector which the one hot code just indexes into.  This can be done as efficiently as this latent vector approach.  The supplementaries contain some really interesting additional information.  I appreciated the plots showing the sinc fit of various other approximations for Bayesian neural nets.  I have done very similar experiments myself and definitely agree that the uncertainty estimates from these is not of high enough quality to use in Bayesian optimization.  However, in the fit of the sinc function shown in Figure 1, you can see that the posterior mean of the function dips down as it gets farther from the data on the left side but up on the right side.  This seems like a result of the highly non-stationary model and an undesirable quality for Bayesian optimization.  For example, the optimization routine (assuming we are minimizing) would arbitrarily explore on the left side instead of the right side.  This doesn't seem to be as much of an issue for the other approaches, as their means seem to asymptote towards the same value on either side.  It would be nice to have an understanding of why this happens and why it might not be a problem.  In the supplementaries it looks like DNGO outperforms SGHMCBO on at least one of the benchmarks (Hartmann 3).  I'm curious why it seems to perform better on this problem.  Does SGHMCBO perhaps do better as the dimensionality of the problem increases (and thus there are more model parameters)?  Nit picky comments: Line 49: It seems weird to indicate the number of experiments using #i since i is a scalar.  Why not follow convention and just use n, i.e. i=1...n