Spread Divergences
For distributions and with different support, the divergence generally will not exist. We define a spread divergence on modified and and describe sufficient conditions for the existence of such a divergence. We give examples of using a spread divergence to train implicit generative models, including linear models (Principal Components Analysis and Independent Components Analysis) and non-linear models (Deep Generative Networks).
 Summary ======= This paper introduces spread divergences. Spread divergences are obtained by taking the divergence between smoothed/noisy versions of two distributions. A spread divergence between two distributions of non-overlapping support can be defined even when the corresponding divergence is not. The authors discuss conditions under which the data generating process can be identified by minimizing spread divergences and apply spread divergences to the examples of PCA, ICA, and noiseless VAE. Review ====== With a lot of papers focusing on generative modeling, divergence minimization is of great relevance to the ICLR community. Adding noise to distributions to ensure overlapping support is intuitive and has been used to stabilize training of GANs, but I am unaware of any work focusing on questions of identifiability and efficiency. I especially like the example of slowing EM in ICA with small noise. Here, some empirical results are lacking which analyze the speed/correctness of the identification of parameters for various choices of divergence/model noise. These would have greatly enhanced the paper. Instead, the available space was used to show model samples, which I find less helpful. In Section 3.2 and Section 6 the authors argue that choosing noise which maximizes the spread divergence is optimal or at least preferable. This seems counterintuitive, given that the point of the noise was to make the distributions more similar. Please elaborate on why maximizing the divergence is a good strategy. Minor ===== The paper seems hastily written, with some grammar issues, typos, and sloppy use of LaTeX, e.g.: – "-.log" instead of ".log" in definition of KL divergence in the introduction – "Section 2.2" not "section(2.2)", "Equation 24" not "equation(24)" – "model (Tipping & Bishop, 1999)" instead of "model Tipping & Bishop (1999)" – ".mid" instead of "|" – "x" instead of "y" in Equation 28 Please provide a reference for the EM algorithm of ICA.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Spread Divergences
For distributions and with different support, the divergence generally will not exist. We define a spread divergence on modified and and describe sufficient conditions for the existence of such a divergence. We give examples of using a spread divergence to train implicit generative models, including linear models (Principal Components Analysis and Independent Components Analysis) and non-linear models (Deep Generative Networks).
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 Summary ======= This paper introduces spread divergences. Spread divergences are obtained by taking the divergence between smoothed/noisy versions of two distributions. A spread divergence between two distributions of non-overlapping support can be defined even when the corresponding divergence is not. The authors discuss conditions under which the data generating process can be identified by minimizing spread divergences and apply spread divergences to the examples of PCA, ICA, and noiseless VAE. Review ====== With a lot of papers focusing on generative modeling, divergence minimization is of great relevance to the ICLR community. Adding noise to distributions to ensure overlapping support is intuitive and has been used to stabilize training of GANs, but I am unaware of any work focusing on questions of identifiability and efficiency. I especially like the example of slowing EM in ICA with small noise. Here, some empirical results are lacking which analyze the speed/correctness of the identification of parameters for various choices of divergence/model noise. These would have greatly enhanced the paper. Instead, the available space was used to show model samples, which I find less helpful. In Section 3.2 and Section 6 the authors argue that choosing noise which maximizes the spread divergence is optimal or at least preferable. This seems counterintuitive, given that the point of the noise was to make the distributions more similar. Please elaborate on why maximizing the divergence is a good strategy. Minor ===== The paper seems hastily written, with some grammar issues, typos, and sloppy use of LaTeX, e.g.: – "-.log" instead of ".log" in definition of KL divergence in the introduction – "Section 2.2" not "section(2.2)", "Equation 24" not "equation(24)" – "model (Tipping & Bishop, 1999)" instead of "model Tipping & Bishop (1999)" – ".mid" instead of "|" – "x" instead of "y" in Equation 28 Please provide a reference for the EM algorithm of ICA.
assistant
 Summary ======= This paper introduces spread divergences. Spread divergences are obtained by taking the divergence between smoothed/noisy versions of two distributions. A spread divergence between two distributions of non-overlapping support can be defined even when the corresponding divergence is not. The authors discuss conditions under which the data generating process can be identified by minimizing spread divergences and apply spread divergences to the examples of PCA, ICA, and noiseless VAE. Review ====== With a lot of papers focusing on generative modeling, divergence minimization is of great relevance to the ICLR community. Adding noise to distributions to ensure overlapping support is intuitive and has been used to stabilize training of GANs, but I am unaware of any work focusing on questions of identifiability and efficiency. I especially like the example of slowing EM in ICA with small noise. Here, some empirical results are lacking which analyze the speed/correctness of the identification of parameters for various choices of divergence/model noise. These would have greatly enhanced the paper. Instead, the available space was used to show model samples, which I find less helpful. In Section 3.2 and Section 6 the authors argue that choosing noise which maximizes the spread divergence is optimal or at least preferable. This seems counterintuitive, given that the point of the noise was to make the distributions more similar. Please elaborate on why maximizing the divergence is a good strategy. Minor ===== The paper seems hastily written, with some grammar issues, typos, and sloppy use of LaTeX, e.g. : – "-.log" instead of ".log" in definition of KL divergence in the introduction – "Section 2.2" not "section(2.2)", "Equation 24" not "equation(24)" – "model (Tipping & Bishop, 1999)" instead of "model Tipping & Bishop (1999)" – ".mid" instead of "|" – "x" instead of "y" in Equation 28 Please provide a reference for the EM algorithm of ICA.