Rényi Divergence Variational Inference
This paper introduces the variational Rényi bound (VR) that extends traditional variational inference to Rényi's α-divergences. This new family of variational methods unifies a number of existing approaches, and enables a smooth interpolation from the evidence lower-bound to the log (marginal) likelihood that is controlled by the value of α that parametrises the divergence. The reparameterization trick, Monte Carlo approximation and stochastic optimisation methods are deployed to obtain a tractable and unified framework for optimisation. We further consider negative α values and propose a novel variational inference method as a new special case in the proposed framework. Experiments on Bayesian neural networks and variational auto-encoders demonstrate the wide applicability of the VR bound.
Paper Summary ============= Conventional variational inference finds an approximate posterior that minimizes the KL divergence from the true (intractable) posterior. This paper shows how variational optimization objectives can use a wider class of Renyi's alpha divergences, which have a scalar parameter \alpha. Setting \alpha appropriately recovers many special cases of interest: \alpha -> 1 recovers the KL divergence while \alpha = 0 leads to exact posterior inference *if* the support of the approximating family contains the true posterior. Negative \alpha values provide an *upper* bound on marginal likelihood and tend to be "mass covering" as \alpha -> -infty, while postive \alpha values provide a lower-bound on the marginal likelihood and tend to be "mode seeking" as \alpha -> +infty.  Aside from connecting Renyi's alpha-divergences to variational objectives (which I haven't seen before), this paper presents a useful algorithm for optimization of these objective functions based on Monte Carlo (MC) approximations similar to those used by Stan and black-box VI [11,12].  The algorithm works by drawing a handful of samples from the current *reparameterized* approximate posterior, computing importance weights for each sample, selecting one sample with probability according to its weight, and then stepping in the direction of its gradient. Special care must apply to minibatch learning, because samples are biased for values of \alpha other than \alpha = 1. Experiments show that alpha values often need to be tuned in a dataset-specific way for best performance, but that values other than alpha=1 are sometimes much better.
Review Summary ============== Overall, I recommend accepting this paper for at least a poster presentation. It nicely unifies and extends various variational inference approaches, and provides a good mix of theoretical and experimental contributions with some solid explanatory figures. My biggest concerns is a lack of practical description of how to properly choose the alpha value for a specific dataset, and how much this can matter compared to other hyperparameters that might need tuning (# of MC samples, sophistication of posterior approximation family etc). I worry that the question of "which alpha to use" will cause concern for practioners and will often result in setting alpha=1 in practice, which is the special case of KL-divergence we already know a lot about. I'd also like to see discussion of how different choices impact computational speed.   Feedback on experiments ======================= My biggest concern is that choosing a good alpha value seems to need careful tuning for each dataset, and there isn't much guidance here on how to do it. It's not clear how much benefit we can get from properly tuning alpha, compared to say fixing alpha=1 (standard VI) and investing precious research time in other choices like using a larger number of samples for MC approximations, tuning minibatch sizes or learning rate schedules, or pursuing a slightly more sophisticated approximate posterior family. From a practical perspective, it would be nice to have a good default guess for alpha and some better intuition about how to tune it / when the default guess is "good enough".  I found the overall story of the first experiment (with Bayesian neural networks) a bit confusing. I would have expected a relatively smooth quadratic-like trend in how the chosen alpha value affects performance, at least within a single dataset. That is, for any dataset, there is an obvious "best choice" of alpha, with performance getting worse for alpha values further away.  owever, the Protein dataset has quite a weird story, where alpha=0.5 is noticeably worse than alpha = 0 or alpha = 1, and then alpha < 0 is worse again. Is there any explanation here? Might this be a local optima issue? In the supplement, the Energy and Yacht datasets have a similarly confusing story.  I wish there was more practical discussion about how various algorithmic choices (# of samples K, alpha value, etc) impact the runtime of the algorithm.I'd be interested in both big-oh and (more importantly) wallclock time comparisons in the experiments. How important is it to just pick one MC sample to do the back-propagation, instead of using all of them?    Feedback on Presentation ======================== I found Fig 2a (which gives intuition for the theorems quanitifying the bias in the MC approximations) quite hard to parse without lots of effort. It's not clear what the arrows mean. I'd suggest expanding this one figure into several smaller panels, each of which explores the influence of one parameter (# of samples K increasing, or alpha decreasing).  Also, some other labeling like alpha_{+} or alpha_{-} might be more useful than alpha_1 and alpha_2 to visually emphasize the domains of these alpha values better.  The technical discussion in Sec 4.3 explaining how to handle minibatches was quite confusing and hard to understand. I'd suggest a careful revision.  Typos, etc ========== Fig 1 caption was hard to parse.  Lines 192-202: Definitions of global and local as used here are a bit confusing. Could be clarified, since many readers will think of the global and local parameter classes described in SVI paper, which (I think) is a different sense of global/local.