Localised Generative Flows
We argue that flow-based density models based on continuous bijections are limited in their ability to learn target distributions with complicated topologies, and propose localised generative flows (LGFs) to address this problem. LGFs are composed of stacked continuous mixtures of bijections, which enables each bijection to learn a local region of the target rather than its entirety. Our method is a generalisation of existing flow-based methods, which can be used without modification as the basis for an LGF model. Unlike normalising flows, LGFs do not permit exact computation of log likelihoods, but we propose a simple variational scheme that performs well in practice. We show empirically that LGFs yield improved performance across a variety of common density estimation tasks.
The authors propose to extend flow-based density models by replacing a single bijection with a hierarchical mixture of bijections. Each component in the mixture is then required to only push the prior onto a local region; this helps improve the coverage of . This is motivated by the conjecture that in many cases the topology of might be overly complicated to be effectively captured by a single bijection. Formally, this is achieved by introducing a conditional random variable . In doing however, the log-likelihood is rendered intractable and a variational approximation must instead be resorted to. A recursive formula for computing the ELBO is introduced in this vain. Overall, I think this is a generally interesting contribution to the normalizing-flow literature that I expect to spark further research. However, there are some rough edges to this paper. The initial motivation is well-presented and relatively easy to follow, though a diagram would serve to cement the intuition regarding the support mismatch. The issue mentioned in footnote 2 deserves further discussion. At the same time, while well-reasoned, their justifications are nonetheless largely conjectural and further theoretical or empirical evidence would be welcome, both for characterising the pathologies they aim to redress and their proposed solution. For the experiment section, I would have liked to have seen comparisons not only to the simplest baseline but also to some of the other methods mentioned in related works. In general, the experiment section is quite short and I didn*t get a very good sense of how well this method performs. The following should be addressed: - provide more evidence for the conjectures surrounding the motivation and derivation - supply more varied baselines (e.g. RAD model) Minor comments: - at the top of page 4, you refer to several times, but I think you mean - in the paragraph after equation (2), the superscript seems to be missing from the - in the first sentence of section 3.1, you refer to *the single used in equation 2*, but equation 2 mentions no - in the first line on page 3, you talk about some region of supp being pushed out of supp , shouldn*t this be the other way round since the KL is infinite only if the the support of is not contained within the support of ?
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Localised Generative Flows
We argue that flow-based density models based on continuous bijections are limited in their ability to learn target distributions with complicated topologies, and propose localised generative flows (LGFs) to address this problem. LGFs are composed of stacked continuous mixtures of bijections, which enables each bijection to learn a local region of the target rather than its entirety. Our method is a generalisation of existing flow-based methods, which can be used without modification as the basis for an LGF model. Unlike normalising flows, LGFs do not permit exact computation of log likelihoods, but we propose a simple variational scheme that performs well in practice. We show empirically that LGFs yield improved performance across a variety of common density estimation tasks.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The authors propose to extend flow-based density models by replacing a single bijection with a hierarchical mixture of bijections. Each component in the mixture is then required to only push the prior onto a local region; this helps improve the coverage of . This is motivated by the conjecture that in many cases the topology of might be overly complicated to be effectively captured by a single bijection. Formally, this is achieved by introducing a conditional random variable . In doing however, the log-likelihood is rendered intractable and a variational approximation must instead be resorted to. A recursive formula for computing the ELBO is introduced in this vain. Overall, I think this is a generally interesting contribution to the normalizing-flow literature that I expect to spark further research. However, there are some rough edges to this paper. The initial motivation is well-presented and relatively easy to follow, though a diagram would serve to cement the intuition regarding the support mismatch. The issue mentioned in footnote 2 deserves further discussion. At the same time, while well-reasoned, their justifications are nonetheless largely conjectural and further theoretical or empirical evidence would be welcome, both for characterising the pathologies they aim to redress and their proposed solution. For the experiment section, I would have liked to have seen comparisons not only to the simplest baseline but also to some of the other methods mentioned in related works. In general, the experiment section is quite short and I didn*t get a very good sense of how well this method performs. The following should be addressed: - provide more evidence for the conjectures surrounding the motivation and derivation - supply more varied baselines (e.g. RAD model) Minor comments: - at the top of page 4, you refer to several times, but I think you mean - in the paragraph after equation (2), the superscript seems to be missing from the - in the first sentence of section 3.1, you refer to *the single used in equation 2*, but equation 2 mentions no - in the first line on page 3, you talk about some region of supp being pushed out of supp , shouldn*t this be the other way round since the KL is infinite only if the the support of is not contained within the support of ?
assistant
The authors propose to extend flow-based density models by replacing a single bijection with a hierarchical mixture of bijections. Each component in the mixture is then required to only push the prior onto a local region; this helps improve the coverage of . This is motivated by the conjecture that in many cases the topology of might be overly complicated to be effectively captured by a single bijection. Formally, this is achieved by introducing a conditional random variable . In doing however, the log-likelihood is rendered intractable and a variational approximation must instead be resorted to. A recursive formula for computing the ELBO is introduced in this vain. Overall, I think this is a generally interesting contribution to the normalizing-flow literature that I expect to spark further research. However, there are some rough edges to this paper. The initial motivation is well-presented and relatively easy to follow, though a diagram would serve to cement the intuition regarding the support mismatch. The issue mentioned in footnote 2 deserves further discussion. At the same time, while well-reasoned, their justifications are nonetheless largely conjectural and further theoretical or empirical evidence would be welcome, both for characterising the pathologies they aim to redress and their proposed solution. For the experiment section, I would have liked to have seen comparisons not only to the simplest baseline but also to some of the other methods mentioned in related works. In general, the experiment section is quite short and I didn*t get a very good sense of how well this method performs. The following should be addressed: - provide more evidence for the conjectures surrounding the motivation and derivation - supply more varied baselines (e.g. RAD model) Minor comments: - at the top of page 4, you refer to several times, but I think you mean - in the paragraph after equation (2), the superscript seems to be missing from the - in the first sentence of section 3.1, you refer to *the single used in equation 2*, but equation 2 mentions no - in the first line on page 3, you talk about some region of supp being pushed out of supp , shouldn*t this be the other way round since the KL is infinite only if the the support of is not contained within the support of ?