Generative Adversarial Networks have made data generation possible in various use cases, but in case of complex, high-dimensional distributions it can be difficult to train them, because of convergence problems and the appearance of mode collapse. Sliced Wasserstein GANs and especially the application of the Max-Sliced Wasserstein distance made it possible to approximate Wasserstein distance during training in an efficient and stable way and helped ease convergence problems of these architectures. This method transforms sample assignment and distance calculation into sorting the one-dimensional projection of the samples, which results a sufficient approximation of the high-dimensional Wasserstein distance. In this paper we will demonstrate that the approximation of the Wasserstein distance by sorting the samples is not always the optimal approach and the greedy assignment of the real and fake samples can result faster convergence and better approximation of the original distribution.
This paper proposes two alternative approaches to max-sliced Wasserstein GANs. They are based on the authors’ claim that there is a “flaw” in the Wasserstein-1 distance between probability distributions on the one-dimensional space. Briefly, the authors’ argument says that the “flaw” is that the optimal transport may not be unique, some of which are better for network learning than others. One proposal, described in Section 2.2, is to find a plausible transport plan in a greedy manner. The other proposal, described in Section2.3, is a hybrid of the greedy approach in Section 2.2 and the original sliced Wasserstein distance.
The working hypothesis in this paper, that the above “flaw” is indeed problematic in learning in max-sliced Wasserstein GANs, has not been confirmed in any sense in this paper. Algorithm 1 is meant to explain one of the proposal, the greedy approach, but I found that several undefined symbols are used there, so that it seems hard to understand it. Numerical experiments in Section 3 are not well described. Because of these, I would not be able to recommend acceptance of this paper.
Algorithm 1 seems to heavily rely on Algorithm 1 in Deshpande et al., 2019. This paper does not provide any explanation about why one should sample n data for each i running from 1 to n (line 3), what the “surrogate loss” is (line 4), what ¥omega is (line 4), why one should care about the surrogate loss between ith data and ith generated sample (line 4), as well as what D^i_k means (line 11).
I do not understand how the Pearson correlation coefficient between the generator (or fake) distribution and the real distribution in Section 3. As far as my understanding, fake samples and real samples are sampled independently, so that correlation coefficient should ideally vanish in any case. Also, the KL divergence is not symmetric, so that whether KL(P_F,P_R) or KL(P_R,P_F) was evaluated has to be explicitly stated. Furthermore, recalling that the Wasserstein distance has originally been introduced to the GAN literature in order to alleviate the problems associated with the KL-based divergence (Jensen-Shannon), I do not understand either why the authors chose to use the KL divergence in their performance comparison.
The “flaw” argued in this paper does not apply to the Wasserstein distance in general, but specifically to the Wasserstein-1 distance between one-dimensional distributions. This fact should be stated clearly.
Page 2, equation (2): The subscript ¥mathbb{P} should read p.
Page 2, two lines below equation (3): w should be italicized.
Page 4, line 4: if the(y->ir) probability
Page 4, line 24: has no effect (t)on the inference complexity
Page 5, Algorithm 1, line 17: g of ¥theta g should be a subscript of ¥theta.
Page 6, line 3: which signals th(e->at) the
Page 6, line 12: was executed for (500.000->500,000) iterations.
Page 6, line 15: Figure number is missing.
Page 7, lines 9-10: (10.000->10,000) random projection(s) and used