AUTO-ENCODING SEQUENTIAL MONTE CARLO
We build on auto-encoding sequential Monte Carlo (AESMC):
1 a method for model and proposal learning based on maximizing the lower bound to the log marginal likelihood in a broad family of structured probabilistic models. Our approach relies on the efficiency of sequential Monte Carlo (SMC) for performing inference in structured probabilistic models and the flexibility of deep neural networks to model complex conditional probability distributions. We develop additional theoretical insights and experiment with a new training procedure which can improve both model and proposal learning. We demonstrate that our approach provides a fast, easy-to-implement and scalable means for simultaneous model learning and proposal adaptation in deep generative models.
Update:
On further consideration (and reading the other reviews), I'm bumping my rating up to a 7. I think there are still some issues, but this work is both valuable and interesting, and it deserves to be published (alongside the Naesseth et al. and Maddison et al. work).
-----------
This paper proposes a version of IWAE-style training that uses SMC instead of classical importance sampling. Going beyond the several papers that proposed this simultaneously, the authors observe a key issue: the variance of the gradient of these IWAE-style bounds (w.r.t. the inference parameters) grows with their accuracy. They therefore propose using a more-biased but lower-variance bound to train the inference parameters, and the more-accurate bound to train the generative model.
Overall, I found this paper quite interesting. There are a few things I think could be cleared up, but this seems like good work (although I'm not totally up to date on the very recent literature in this area).
Some comments:
* Section 4: I found this argument extremely interesting. However, it’s worth noting that your argument implies that you could get an O(1) SNR by averaging K noisy estimates of I_K. Rainforth et al. suggest this approach, as well as the approach of averaging K^2 noisy estimates, which the theory suggests may be more appropriate if the functions involved are sufficiently smooth, which even for ReLU networks that are non-differentiable at a finite number of points I think they should be.
This paper would be stronger if it compared with Rainforth et al.’s proposed approaches. This would demonstrate the real tradeoffs between bias, variance, and computation. Of course, that involves O(K^2) or O(K^3) computation, which is a weakness. But one could use a small value of K (say, K=5).
That said, I could also imagine a scenario where there is no benefit to generating multiple noisy samples for a single example versus a single noisy sample for multiple examples. Basically, these all seem like interesting and important empirical questions that would be nice to explore in a bit more detail.
* Section 3.3: Claim 1 is an interesting observation. But Propositions 1 and 2 seem to just say that the only way to get a perfectly tight SMC ELBO is to perfectly sample from the joint posterior. I think there’s an easier way to make this argument:
Given an unbiased estimator \hat{Z} of Z, by Jensen’s inequality E[log \hat{Z}] ≤ log Z, with equality iff the variance of \hat{Z} = 0. The only way to get an SMC estimator’s variance to 0 is to drive the variance of the weights to 0. That only happens if you perfectly sample each particle from the true posterior, conditioned on all future information.
All of which is true as far as it goes, but I think it’s a bit of a distraction. The question is not “what’s it take to get to 0 variance” but “how quickly can we approach 0 variance”. In principle IS and SMC can achieve arbitrarily high accuracy by making K astronomically large. (Although [particle] MCMC is probably a better choice if one wants extremely low bias.)
* Section 3.2: The choice of how to get low-variance gradients through the ancestor-sampling choice seems seems like an important technical challenge in getting this approach to work, but there’s only a very cursory discussion in the main text. I would recommend at least summarizing the main findings of Appendix A in the main text.
* A relevant missing citation: Turner and Sahani’s “Two problems with variational expectation maximisation for time-series models” (http://www.gatsby.ucl.ac.uk/~maneesh/papers/turner-sahani-2010-ildn.pdf). They discuss in detail some examples where tighter variational bounds in state-space models lead to worse parameter estimates (though in a quite different context and with a quite different analysis).
* Figure 1: What is the x-axis here? Presumably phi is not actually 1-dimensional?
Typos etc.:
* “learn a particular series intermediate” missing “of”.
* “To do so, we generate on sequence y1:T” s/on/a/, I think?
* Equation 3: Should there be a (1/K) in Z?