Hamiltonian Variational Auto-Encoder
Variational Auto-Encoders (VAEs) have become very popular techniques to perform inference and learning in latent variable models: they allow us to leverage the rich representational power of neural networks to obtain flexible approximations of the posterior of latent variables as well as tight evidence lower bounds (ELBOs). Combined with stochastic variational inference, this provides a methodology scaling to large datasets. However, for this methodology to be practically efficient, it is necessary to obtain low-variance unbiased estimators of the ELBO and its gradients with respect to the parameters of interest. While the use of Markov chain Monte Carlo (MCMC) techniques such as Hamiltonian Monte Carlo (HMC) has been previously suggested to achieve this [25,28], the proposed methods require specifying reverse kernels which have a large impact on performance. Additionally, the resulting unbiased estimator of the ELBO for most MCMC kernels is typically not amenable to the reparameterization trick. We show here how to optimally select reverse kernels in this setting and, by building upon Hamiltonian Importance Sampling (HIS) [19], we obtain a scheme that provides low-variance unbiased estimators of the ELBO and its gradients using the reparameterization trick. This allows us to develop a Hamiltonian Variational Auto-Encoder (HVAE). This method can be re-interpreted as a target-informed normalizing flow [22] which, within our context, only requires a few evaluations of the gradient of the sampled likelihood and trivial Jacobian calculations at each iteration.
I am satisfied with the authors' reply, especially about the clarification of volume preservation and Hamiltonian conservation. The additional experiments are also promising.   I raise my score to 7 and agree to accept this paper.   *** original review ********  In this paper, the authors exploit the Hamiltonian sampling updates to construct the posterior family. Due to the volume is preserved asymptotically, the posterior can be explicitly represented, which leads to closed form of the entropy term in the ELBO. With such property, the gradient w.r.t. the parameters of both posterior and the graphical model can be calculated through the Hamiltonian updates, i.e, reverse-mode differentiation trick [5, 6], and thus, stochastic gradient descent can be utilized for parameters updating. The authors tested the proposed algorithm on a simple Gaussian model on sythetic dataset and the generative model on MNIST to demonstrate the benefits of the proposed algorithm.   There are three major issues the authors should addressed.  1, The volume preservation property only holds asymptotically, i.e., the stepsize goes to zero. When introducing the discretization step, i.e., the leapfrog update, such property no longer holds. (Otherwise, there is no need the MCMC step in the Hybrid Monte-Carlo algorithm) Therefore, the claim in line 145-146, "the composition of equations (7)-(9) has unit Jacobian since each quation describes a shear transformation", is not correct. There will be extra error introduced due to the discretization, which leads to the posterior formulation in the paper is only an approximation.   Please correct such misleading claim and disucss the effect of discretization in details.   2, There are severl important references missing. As far as I know, the reverse-mode differentiation (RMD) [5, 6] has been applied to variational inference [1, 2, 3, 4]. (These are only partial work in this direction.) None of such closely related work has been cited, discussed, and empirically compared.   Please check the liturature and cite the related work appropriately.  Moreover, the RMD has explict flaw that the memory cost will be huge if there are many iterations of the Hamiltonian update.   Please discuss such effect in the paper too.  3, The experiment part is weak. The authors only compared to vanilla VAE with CNN on MNIST. The other state-of-the-art variational inference methods, e.g., normalized flow, adversarial variational Bayes, and the most related one proposed in [3, 4, 7]. Without the comprehensive comparison with these algorithms, it is not supportive to claim the benefits of the proposed algorithm.   In sum, I like the idea to exploit the Hamiltonian updates to construct posterior. However, this paper can be improved in many places.  [1] Justin Domke. Generic methods for optimization-based modeling. In Artificial Intelligence and Statistics, pages 318–326, 2012.  [2] Veselin Stoyanov, Alexander Ropson, and Jason Eisner. Empirical risk minimization of graphical model parameters given approximate inference, decoding, and model structure. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pages 725–733, 2011.  [3] Yisong Yue, Stephan Mandt, and Joseph Marino. Iterative Amortized Inference. International Conference on Machine Learning (ICML), July 2018.  [4] Kim, Y., Wiseman, S., Miller, A. C., Sontag, D., and Rush, A. M. Semi-amortized variational autoencoders. In Proceedings of the International Conference on Machine Learning (ICML), 2018.   [5] Atilim Gunes Baydin and Barak A Pearlmutter. Automatic differentiation of algorithms for machine learning. arXiv preprint arXiv:1404.7456, 2014.  [6] Yoshua Bengio. Gradient-based optimization of hyperparameters. Neural computation, 12(8):1889–1900, 2000.  [7] Tim Salimans, Diederik P Kingma, and Max Welling. Markov chain Monte Carlo and variational inference: Bridging the gap. In International Conference on Machine Learning, pages 1218–1226, 2015.