The importance weighted autoencoder (IWAE) (Burda et al., 2016) is a popular variational-inference method which achieves a tighter evidence bound (and hence a lower bias) than standard variational autoencoders by optimising a multi-sample objective, i.e. an objective that is expressible as an integral over K >
UPDATE: bumping up my score after the revisions
---
Nice connections but novelty and practical takeaways unclear
SUMMARY OF THE PAPER:
This paper views recent IWAE-based [1] methods (IWAE-STL [2], IWAE-DREG [3], RWS [4, 5]) for training generative models p and inference networks q under a common framework, AISLE.
This heavily relies on the "double-reparameterization" property by [2] and is restated in Lemma 1.
This framework makes it explicit that we're interested in
1) maximizing the (log) marginal likelihood wrt p parameters, and
2) minimizing some divergence between the posterior in the learned model to q.
In AISLE, IWAE-STL's q-gradient is viewed as a doubly-reparameterized self-normalized importance sampling (SNIS) estimate of KL(p || q).
This is in contrast to viewing it as a biased estimator of the IWAE's q-gradient.
This can potentially explain why it performs well when number of SNIS samples are increased.
It is also some evidence against the fact that having no unified objective is bad (because there isn't evidence of IWAE-STL diverging despite there being no unified objective).
IWAE-DREG's q-gradient is viewed as a doubly-reparameterized SNIS estimate of X-divergence(p || q) (up to multiplicative constant of the number of SNIS samples).
This is in contrast to viewing it as an unbiased estimator of the IWAE's q-gradient.
The view on RWS is unchanged: the q-gradient is a SNIS estimate of KL(p || q).
For me, the main contribution is viewing IWAE-STL and IWAE-DREG q-gradient estimators as biased gradients of an explicit divergence rather than of the IWAE objective.
I also found the observation that the signal-to-noise (SNR) decrease in IWAE's q-gradient can be proved by noting that it is a SNIS estimator of a zero vector nice.
STRUCTURE:
The article is well-written and easy to understand.
NOVELTY:
A different view on IWAE-STL and IWAE-DREG is interesting and novel (as mentioned above).
This means that IWAE-STL and IWAE-DREG are good not only because they reduce gradient variance (as previously understood) but also potentially because they directly target a divergence.
Viewing generalization of RWS as a main contribution (first bulletpoint of Section 1.2: "...we show that AISLE admits RWS as a special case.") is a bit of a stretch since this generalization is very straightforward from the way RWS is formulated.
The recommendation of using RWS-style algorithms over IWAE as given in the abstract ("we argue that directly optimising the proposal distribution in importance sampling as in the RWS algorithm is preferable to optimising IWAE-type multi-sample objectives) is also not novel since this is also advocated by [5] (section 3.2: "This makes RWS a preferable option to IWAE for learning inference networks because the phi updates in RWS directly target minimization of the expected KL divergences from the true to approximate posterior").
The recommendation as a method for non-reparameterisable latent variables at the end of section 1.2 ("as well as further algorithms which do not require reparameterisations") is also given in [5].
Are there different adaptive importance sampling algorithms that could be used within AISLE that would improve on IWAE-STL/IWAE-DREG/RWS?
EXPERIMENTS:
There are no experiments in the main paper.
However, experiments that would support/falsify the following points could be good:
- RWS and IWAE-STL don't suffer from non-unified objectives because IWAE-STL has non-unified objectives but doesn't diverge,
- [targeting direct divergence] is more useful than (or as useful as) [lower variance gradient estimators].
CONCLUSION:
While I really like the presentation and connections made in the paper, I'm not sure what the practical takeaways are (other than use IWAE-STL, IWAE-DREG, RWS over IWAE which is advocated by [2], [3], [5]).
I'm giving this a weak accept due to the former.
I'm willing to bump up my score if
- the paper is modified to more accurately reflect the contributions or
- there are experiments that provide additional support for the [targeting direct divergence] view in addition to [2, 3, 4, 5], or
- there is a new practical algorithm that the AISLE generalization would suggest that is better than IWAE-STL, IWAE-DREG, RWS in some respects.
[1] Importance Weighted Autoencoders. https://arxiv.org/abs/1509.00519
[2] Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference. https://arxiv.org/abs/1703.09194
[3] Doubly Reparameterized Gradient Estimators for Monte Carlo Objectives. https://arxiv.org/abs/1810.04152
[4] Reweighted Wake-Sleep. https://arxiv.org/abs/1406.2751
[5] Revisiting Reweighted Wake-Sleep for Models with Stochastic Control Flow. https://arxiv.org/abs/1805.10469
[6] Variational Inference via χ-Upper Bound Minimization. https://arxiv.org/abs/1611.00328
[7] Tighter Variational Bounds are Not Necessarily Better. https://arxiv.org/abs/1802.04537