Generalizing Hamiltonian Monte Carlo with Neural Networks
We present a general-purpose method to train Markov chain Monte Carlo kernels, parameterized by deep neural networks, that converge and mix quickly to their target distribution. Our method generalizes Hamiltonian Monte Carlo and is trained to maximize expected squared jumped distance, a proxy for mixing speed. We demonstrate large empirical gains on a collection of simple but challenging distributions, for instance achieving a 106x improvement in effective sample size in one case, and mixing when standard HMC makes no measurable progress in a second. Finally, we show quantitative and qualitative gains on a real-world task: latent-variable generative modeling. Python source code will be open-sourced with the camera-ready paper.
 The paper introduces a non-volume-preserving generalization of HMC whose transitions are determined by a set of neural network functions. These functions are trained to maximize expected squared jump distance. This works because each variable (of the state space) is modified in turn, so that the resulting update is invertible, with a tractable transformation inspired by Dinh et al 2016. Overall, I believe this paper is of good quality, clearly and carefully written, and potentially accelerates mixing in a state-of-the-art MCMC method, HMC, in many practical cases. A few downsides are commented on below. The experimental section proves the usefulness of the method on a range of relevant test cases; in addition, an application to a latent variable model is provided sec5.2. Fig 1a presents results in terms of numbers of gradient evaluations, but I couldn*t find much in the way of computational cost of L2HMC in the paper. I can*t see where the number "124x" in sec 5.1 stems from. As a user, I would be interested in the typical computational cost of both "MCMC sampler training" and MCMC sampler usage (inference?), compared to competing methods. This is admittedly hard to quantify objectively, but just an order of magnitude would be helpful for orientation. Would it be relevant, in sec5.1, to compare to other methods than just HMC, eg LAHMC? I am missing an intuition for several things: eq7, the time encoding defined in Appendix C Appendix Fig5, I cannot quite see how the caption claim is supported by the figure (just hardly for VAE, but not for HMC). The number "124x ESS" in sec5.1 seems at odds with the number in the abstract, "50x". # Minor errors - sec1: "The sampler is trained to minimize a variation": should be maximize "as well as on a the real-world" - sec3.2 "and 1/2 v^T v the kinetic": "energy" missing - sec4: the acronym L2HMC is not expanded anywhere in the paper The sentence "We will denote the complete augmented...p(d)" might be moved to after "from a uniform distribution" in the same paragraph. In paragraph starting "We now update x": - specify for clarity: "the first update, which yields x* "/ "the second update, which yields x** " - "only affects ": should be (prime missing) - the syntax using subscript m^t is confusing to read; wouldn*t it be clearer to write this as a function, eg "mask(x*,m^t)"? - inside zeta_2 and zeta_3, do you not mean .bar{m}^t$ ? - sec5: add reference for first mention of "A NICE MC" - Appendix A: - "Let*s" -> "Let" - eq12 should be x**=... - Appendix C: space missing after "Section 5.1" - Appendix D1: "In this section is presented" : sounds odd - Appendix D3: presumably this should consist of the figure 5 ? Maybe specify.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Generalizing Hamiltonian Monte Carlo with Neural Networks
We present a general-purpose method to train Markov chain Monte Carlo kernels, parameterized by deep neural networks, that converge and mix quickly to their target distribution. Our method generalizes Hamiltonian Monte Carlo and is trained to maximize expected squared jumped distance, a proxy for mixing speed. We demonstrate large empirical gains on a collection of simple but challenging distributions, for instance achieving a 106x improvement in effective sample size in one case, and mixing when standard HMC makes no measurable progress in a second. Finally, we show quantitative and qualitative gains on a real-world task: latent-variable generative modeling. Python source code will be open-sourced with the camera-ready paper.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 The paper introduces a non-volume-preserving generalization of HMC whose transitions are determined by a set of neural network functions. These functions are trained to maximize expected squared jump distance. This works because each variable (of the state space) is modified in turn, so that the resulting update is invertible, with a tractable transformation inspired by Dinh et al 2016. Overall, I believe this paper is of good quality, clearly and carefully written, and potentially accelerates mixing in a state-of-the-art MCMC method, HMC, in many practical cases. A few downsides are commented on below. The experimental section proves the usefulness of the method on a range of relevant test cases; in addition, an application to a latent variable model is provided sec5.2. Fig 1a presents results in terms of numbers of gradient evaluations, but I couldn*t find much in the way of computational cost of L2HMC in the paper. I can*t see where the number "124x" in sec 5.1 stems from. As a user, I would be interested in the typical computational cost of both "MCMC sampler training" and MCMC sampler usage (inference?), compared to competing methods. This is admittedly hard to quantify objectively, but just an order of magnitude would be helpful for orientation. Would it be relevant, in sec5.1, to compare to other methods than just HMC, eg LAHMC? I am missing an intuition for several things: eq7, the time encoding defined in Appendix C Appendix Fig5, I cannot quite see how the caption claim is supported by the figure (just hardly for VAE, but not for HMC). The number "124x ESS" in sec5.1 seems at odds with the number in the abstract, "50x". # Minor errors - sec1: "The sampler is trained to minimize a variation": should be maximize "as well as on a the real-world" - sec3.2 "and 1/2 v^T v the kinetic": "energy" missing - sec4: the acronym L2HMC is not expanded anywhere in the paper The sentence "We will denote the complete augmented...p(d)" might be moved to after "from a uniform distribution" in the same paragraph. In paragraph starting "We now update x": - specify for clarity: "the first update, which yields x* "/ "the second update, which yields x** " - "only affects ": should be (prime missing) - the syntax using subscript m^t is confusing to read; wouldn*t it be clearer to write this as a function, eg "mask(x*,m^t)"? - inside zeta_2 and zeta_3, do you not mean .bar{m}^t$ ? - sec5: add reference for first mention of "A NICE MC" - Appendix A: - "Let*s" -> "Let" - eq12 should be x**=... - Appendix C: space missing after "Section 5.1" - Appendix D1: "In this section is presented" : sounds odd - Appendix D3: presumably this should consist of the figure 5 ? Maybe specify.
assistant
 The paper introduces a non-volume-preserving generalization of HMC whose transitions are determined by a set of neural network functions. These functions are trained to maximize expected squared jump distance. This works because each variable (of the state space) is modified in turn, so that the resulting update is invertible, with a tractable transformation inspired by Dinh et al 2016. Overall, I believe this paper is of good quality, clearly and carefully written, and potentially accelerates mixing in a state-of-the-art MCMC method, HMC, in many practical cases. A few downsides are commented on below. The experimental section proves the usefulness of the method on a range of relevant test cases; in addition, an application to a latent variable model is provided sec5.2. Fig 1a presents results in terms of numbers of gradient evaluations, but I couldn*t find much in the way of computational cost of L2HMC in the paper. I can*t see where the number "124x" in sec 5.1 stems from. As a user, I would be interested in the typical computational cost of both "MCMC sampler training" and MCMC sampler usage (inference? ), compared to competing methods. This is admittedly hard to quantify objectively, but just an order of magnitude would be helpful for orientation. Would it be relevant, in sec5.1, to compare to other methods than just HMC, eg LAHMC? I am missing an intuition for several things: eq7, the time encoding defined in Appendix C Appendix Fig5, I cannot quite see how the caption claim is supported by the figure (just hardly for VAE, but not for HMC). The number "124x ESS" in sec5.1 seems at odds with the number in the abstract, "50x". # Minor errors - sec1: "The sampler is trained to minimize a variation": should be maximize "as well as on a the real-world" - sec3.2 "and 1/2 v^T v the kinetic": "energy" missing - sec4: the acronym L2HMC is not expanded anywhere in the paper The sentence "We will denote the complete augmented...p(d)" might be moved to after "from a uniform distribution" in the same paragraph. In paragraph starting "We now update x": - specify for clarity: "the first update, which yields x* "/ "the second update, which yields x** " - "only affects ": should be (prime missing) - the syntax using subscript m^t is confusing to read; wouldn*t it be clearer to write this as a function, eg "mask(x*,m^t)"? - inside zeta_2 and zeta_3, do you not mean .bar{m}^t$ ? - sec5: add reference for first mention of "A NICE MC" - Appendix A: - "Let*s" -> "Let" - eq12 should be x**=... - Appendix C: space missing after "Section 5.1" - Appendix D1: "In this section is presented" : sounds odd - Appendix D3: presumably this should consist of the figure 5 ? Maybe specify.