Using Large Ensembles of Control Variates for Variational Inference
Variational inference is increasingly being addressed with stochastic optimization. In this setting, the gradient's variance plays a crucial role in the optimization procedure, since high variance gradients lead to poor convergence. A popular approach used to reduce gradient's variance involves the use of control variates. Despite the good results obtained, control variates developed for variational inference are typically looked at in isolation. In this paper we clarify the large number of control variates that are available by giving a systematic view of how they are derived. We also present a Bayesian risk minimization framework in which the quality of a procedure for combining control variates is quantified by its effect on optimization convergence rates, which leads to a very simple combination rule. Results show that combining a large number of control variates this way significantly improves the convergence of inference over using the typical gradient estimators or a reduced number of control variates.
Thank you for the thoughtful response.  I have read the other reviews and the rebuttal, and after discussing the work I am electing to keep my score the same.   I am somewhat unsatisfied by the author response; for papers where gradient estimator efficiency (in terms of variance) is in service of the optimization problem, comparing ELBO traces by iteration can be very misleading.  If the machinery you introduce to efficiently use an ensemble of control variates is not very costly, then it should be measured or shown in your experiments.   My comments below weren't about optimal tuning, they were more about exploring/understanding the sensitivity of their method on the parameters they introduce. As I understand it, the main contribution of this work is the development of a regularized estimator of what is essentially a linear regression coefficient using conjugacy in exponential families.  I think that's a very clever way to approach the problem of combining control variates.  But, from the experiments, I did not get a sense the sensitivity of their estimator to these prior parameters was explored. How were they set? How do they interact with the number of samples chosen to construct this estimator? If this were a model for data, we would expect some sensitivity analysis to be done to see how much prior assumptions are factoring into posterior inferences. Analogously, I would hope to see some sensitivity analysis of the proposed method.  And from the rebuttal, I get the sense that these sensitivities will be left unexplored.    -------------------------------- The authors present a framework for constructing control variates for stochastic optimization in variational settings.  Looking at sub-parts of the variational inference objective, they define broad classes of ways to construct a control variate for each piece.  They then define a way to combine multiple control variates into a single, low-variance estimator in a sample-efficient way using a regret-minimizing estimator.  They compare the performance of these estimators on a logistic regression model with three different data sets at different learning rates.    - How does wall clock time compare for each estimator?  One issue with iteration-based comparisons is that some control variates (or their ensemble) may require more likelihood or gradient calculations.  In table 1 -- how much time did each iteration take for each set of estimators?  Another way to compare might be the number of likelihood evaluations required per iteration.   - Figure 3 is missing a legend --- do the colors correspond to the estimators defined in the legend in Figure 2?   - How does the running average parameter (gamma) from section 4.2 affect the variance of the estimator throughout optimization?  This is an interesting pivot point of computational tradeoff --- fitting the optimal C with many samples at an iteration of optimization would yield the lowest variance estimator.  How does that degrade when you (i) use fewer samples and rely on the InvWishart prior and (ii) use a running average of previous iterates?  At a certain point it becomes not worth computing a "better C" when optimizing an objective --- where is that point?  And how would you go about systematically finding it?  *Quality*: I think this paper presents an interesting idea, and points to a way to use a bunch of existing estimators to get the most out of a stochastic optimization procedure.  I think some of the experiments could have been conducted in a more convincing way, keeping in mind the goal of reducing the overall compute time of the optimization algorithm.  I liked how the experiments section compared a wide variety of estimators, and I would like to see them probe their own contribution a little more --- how their scheme for fitting C is affected by prior assumptions, how much compute time it takes per iteration, and how the running average scheme affects optimization.  *Clarity*: The paper is very clearly written and easy to follow.  *Originality*: The paper compares a bunch of existing estimators within a framework that is somewhat novel.  The regularization and running average of the control variate parameter is an interesting extension and a nice way to combine information from multiple estimators.  However the original aspect of their paper was empirically probed the least.  *Significance*: It is difficult to assess the potential impact of this method given the experiments presented. Ultimately, the utility of these gradient estimators for optimization hinge on wall clock time.