The standard variational lower bounds used to train latent variable models produce biased estimates of most quantities of interest. We introduce an unbiased estimator of the log marginal likelihood and its gradients for latent variable models based on randomized truncation of infinite series. If parameterized by an encoder-decoder architecture, the parameters of the encoder can be optimized to minimize its variance of this estimator. We show that models trained using our estimator give better test-set likelihoods than a standard importance-sampling based approach for the same average computational cost. This estimator also allows use of latent variable models for tasks where unbiased estimators, rather than marginal likelihood lower bounds, are preferred, such as minimizing reverse KL divergences and estimating score functions.
This paper proposes an unbiased estimator of $\log p_\theta(x)$. Many unbiased estimators of $p_\theta$ exist, but $\log p_\theta$ is needed in many other settings, some of which are not well-served by standard estimators of $p_\theta$. The SUMO estimator is essentially a Russian roulette-based extension of IWAE; it is exactly unbiased, but takes a random and unbounded number of samples.
This allows marginally better optimization of certain models than IWAE with a much smaller average number of samples, and (more importantly) opens new possibilities such as entropy maximization which are not well-served by lower bounds like IWAE. This is a very nice advantage of this estimator.
One complaint about this class of estimators in general is: yes, the exact SUMO procedure is technically unbiased. But in practice, if SUMO takes more than, say, a day of compute time – something that will happen with extremely small but nonzero probability – then the user will kill it. And the SUMO estimator conditioned on taking less than a day of compute time is actually biased. This also likely means that, though unbiased, these estimators can be potentially skewed or otherwise "unpleasant." For SUMO in particular, the estimator with $K$ truncated probably has bias bounded based on the bias of IWAE with batch size equal to the truncation point, which is likely quite small. But it would be nice to understand this a little more. (Perhaps it's been studied by some of the recent cited work on these types of estimators.)
Relatedly, you don't prove that this estimator has a finite variance, and in fact it seems plausible theoretically that it might be infinite. Like the "notorious" harmonic mean estimator, this is troubling. It seems that things are okay in practice, but how can we tell whether the variance is really finite or not? I don't know if there's a good answer, but one diagnostic might be something like the one suggested at https://stats.stackexchange.com/a/9143 . (Your comments about occasional "bounded but very large" gradient estimates are troubling in this respect, depending on what exactly you mean by "bounded".) When you do gradient clipping, the estimator of course then has a finite variance, but can we get some sense of how much bias that introduces?
Overall, though, I think this is a very nice new estimator that is both well-founded – despite leaving some questions open – and likely to be practically useful. Given that it is also extremely on-topic for ICLR and novel, I'm rating the paper as "accept."
(For slightly more detail on the "thoroughness assessment": I did not really check the proofs in the appendix, but did pay attention to the derivations in the main body.)
Smaller notes:
- Top of page 3: you comment that SGD "requires unbiased estimates of" gradients of the log-density. In fact, SGD can be shown to work with biased gradient estimators, with suboptimality in the results depending on the bias; see e.g. Chen and Luss, http://arxiv.org/abs/1807.11880 .
- In the definition of $\tilde{Y}$, above (7): it might make more sense to define $\tilde{Y}$ with some recursive scheme, rather than as an estimator that either computes one of the $\Delta$ values or infinitely many of them.
- Start of 3.1: presumably $\Delta_k$ is what converges absolutely, not IWAE?
- Start of 3.2: as you note, it is clearly not true that the $\Delta_k^g$ are independent. But you don't really "assume independence" – the Russian roulette estimator is still a valid estimator, just perhaps not the optimal among that class. It would be better to say something like that since it seems that the $\Delta_k^g$ are *nearly* independent (or at least nearly uncorrelated), the Russian roulette estimator is probably at least a reasonable choice.
- Re: the discussion after (9) and in (12), as well as a few other places: I think you show in Appendix A.3 that $\mathbb E[ \nabla \operatorname{SUMO}(x) ]$ exists, but you don't show that it equals $\nabla \mathbb E[ \operatorname{SUMO}(x) ]$. This is likely true, particularly if $q$ and $p$ are each everywhere-differentiable, and it's totally fine if you don't want to prove it out formally, but it would be worth at least a footnote that this is a thing that requires proof. (See e.g. https://arxiv.org/abs/1801.01401 Theorem 5, for a formal result of this type supporting ReLU activations, which you may be able to just use directly.)