We propose a novel algorithm, Difference-Seeking Generative Adversarial Network (DSGAN), developed from traditional GAN. DSGAN considers the scenario that the training samples of target distribution, p t , are difficult to collect. Suppose there are two distributions pd and p d such that the density of the target distribution can be the differences between the densities of pd and p d . We show how to learn the target distribution p t only via samples from p d and pd (relatively easy to obtain). DSGAN has the flexibility to produce samples from various target distributions (e.g. the out-of-distribution). Two key applications, semi-supervised learning and adversarial training, are taken as examples to validate the effectiveness of DSGAN. We also provide theoretical analyses about the convergence of DSGAN.
The paper presents a new Generative Adversarial Network (GAN) for learning a  
target distribution that is defined as the difference between two other 
distributions. Applications in semi-supervised learning and adversarial training 
are considered in the experimental evaluation and results are presented in 
computer vision tasks. 
The paper is not very well written and can be hard to follow. One very important 
issue for me was motivation for defining the target distribution as a difference 
between two other distributions. I am not familiar with this area, but reading 
through the introduction it was never clear to me why this is a useful scenario, 
in practice. Furthermore, some statements in the introduction felt quite 
arbitrary. For example, the authors state that PixelCNN "does not have a latent 
representation" in a manner that makes it sound as if that is a bad thing. If 
indeed it is, then why so? It would be very helpful to motivate the setting more 
and to provide a couple of examples of where this method would be useful, in the 
introduction. Also, regarding the MNIST example in the end of page 1, what is 
the "universal set"? This paragraph also felt a bit arbitrary and unclear.
Some comments about the rest of the paper:
  - The theoretical results of section 3 are just stated/listed, but are not 
    connected to algorithm 1. Please connect them to the different parts of the 
    algorithm and state in a couple sentences what they imply for the algorithm.
  - Right after theorem 1, which assumption are you referring to when you say 
    "the assumption in Theorem 1"?
  - The reformulation of section 3.1 is never justified. What led you to use 
    this reformulation and why do you think it is more stable in practice?
  - You should mention in the caption of table 4, what quantity you are 
    computing.
Note that my evaluation for this paper is based mainly on the way it is written 
as, in its current state, it is hard for me to judge what is novel and what is 
useful, and what readers are supposed to take in by reading this paper. The main 
question that the paper definitely needs to answer, but does not do so currently 
(in my opinion) is:
  When is this method useful to readers? For solving which problems and under 
  what conditions? And also, when is this method bad and should not be used?
== Experiments ==
Section 5.1 is hard to follow and I don't quite get how it connects to the rest.
Also, in section 5.1.2 you mention that in comparison to Dai et al. (2017) your 
method does not need to rely on an additional density estimation network. Even 
if that is true, I cannot see how it is a useful remark given that the method of 
Dai et al. seems to always beat your method.
== Style ==
In figure 1, no labels or legends are provided making it hard to figure out 
what's going on at a glance. It would be very helpful to include labels and a 
legend.
Equation 2 is not written correctly. The equals sign only refers to "V(G, D)" 
and not the min-max of that, right? Please make that explicit by first defining 
"V(G, D)" alone.