For distributions p and q with different support, the divergence D(p||q) generally will not exist. We define a spread divergenceD(p||q) on modified p and q and describe sufficient conditions for the existence of such a divergence. We give examples of using a spread divergence to train implicit generative models, including linear models (Principal Components Analysis and Independent Components Analysis) and non-linear models (Deep Generative Networks).
Pros:
- interesting idea.
Cons:
- the paper forgets the state of the art for comparisons (optimal transport, data processing inequalities)
- the paper formally shows little as most results are in fact buried in the text and it is hard to tell the formal from the informal.
- experiments fall short of really using the setting proposed
- the paper focuses too much on keeping the identity of the indiscernibles and forgets the study of other properties (including downsides, such as variance increase)
Detail:
* The paper claims to propose a "theory" for spread divergences (conditioning a f-divergence by a "third-party" conditional distribution on supports which makes supports match) still keeping the identity of indiscernibles. 
* The paper recycles the notion of Spread f-divergences from Zhang et al. (which makes a circular reference to this paper for the introduction of these divergences).
* The paper motivates the notion of spread divergences by the fact that f-divergences impose matching supports (Section 1), not mentioning that optimal transport theory is a much natural fit for any such kind of setting (e.g. Wasserstein distances). This is a big omission and a missed occasion for a potentially interesting discussion.
* The paper then claims that "spread noise makes distributions more similar" (Section 2.1), not mentioning that equation (8), which it claims to have been shown by Zhang et al. paper (see below), is in fact a data processing inequality *long known*. They will find it, along with a huge number of other useful properties, in series of IEEE T. IT papers, among which Pardo and Vajda's "About distances of discrete distributions satisfying the data processing theorem of information theory",  Van Erven and Harremoes, "Re ́nyi Divergence and Kullback-Leibler Divergence" (for the KL / Rényi divergence, but you have more references inside), etc. .
* The paper then goes on "showing" (a word used often, even when there is not a single Theorem, Lemma or the like ever stated in the paper...) several properties (Section 2.2). The first states that (9) is equivalent to P being invertible. It is wrong because it is just in fact stating (literally) that P defines an injective mapping. The second states that (11) is equivalent to (12), without the beginning of a proof. I do need to see a proof, and in particular how you "define" an invertible transform "p^-1".
* The paper then gives two examples (Sections 3, 4). In Section 3, I am a bit confused because it seems that p and q must have supports in IR, which limits the scope of the example. The same limitation applies to Section 4, even when it is a bit more interesting. In all cases, the authors must properly state a Lemma in each Section that states and shows what is claimed before Section 3.
* The paper then makes several experiments. Unless I am mistaken, it seems that Section 5.1 relies on a trick that does not change the support from x to y. Therefore, what is the interest of the approach in this case ? In Section 5.2, isn’t the trick equivalent to considering ICA with a larger \gamma ?
* A concern is that the paper says little about the reason why we should pick one p(y|x) instead of another one. The focus is on the identity of indiscernibles. The paper also forgets some potential drawbacks of the technique, including the fact that variance increases — the increase can be important with bad choices, which is certainly not a good thing.