out-of-distribution examples in the MNIST dataset, from another perspective, are found to belong to the differences between the sets of examples in MNIST and the universal set. It should be noted that in traditional GAN, the target distribution is identical to the training data distribution; however, in the DSGAN these two distributions, are considered to be different. This paper makes the following contributions:
(1) We propose the DSGAN to generate any unseen data only if the density of the target (unseen data) distribution is the difference between those of any two distributions, pd and p d . (2) We show that the DSGAN possesses the flexibility to learn different target (unseen data) distributions in two key applications, semi-supervised learning and novelty detection. Specifically, for novelty detection, the DSGAN can produce boundary points around the seen data because this type of unseen data is easily misclassified. For semi-supervised learning, the unseen data are linear combinations of any labeled data and unlabeled data, excluding the labeled and unlabeled data themselves 2 .
(3) The DSGAN yields results comparable to a semi-supervised learning but with a short training time and low memory consumption. In novelty detection, combining both the DSGAN and variational auto-encoder (VAE, Kingma & Welling (2014b)) methods achieve the state-of-the-art results.
This paper proposed DSGAN which learns to generate unseen data from seen data distribution p_d and its somehow “broad” version p_{\hat d} (E.g., p_d convolved with Gaussian). The “unseen data” is the one that appears in p_{\hat d} but not in p_d. DSGAN is trained to generate such data. In particular, it uses samples from p_d as fake data and samples from p_{\hat d} as the real one. 
Although the idea seems to be interesting, the paper seems to be a bit incremental and is a simple application of existing GAN techniques. The paper shows two applications (semi-supervised learning and novelty detection) and it is not clear that the proposed method outperforms existing GAN methods in the classification accuracy in MNIST/SVHN/CIFAR10 (Table 1) and existing sampling methods (Table. 3). It seems that the sampled reconstruction results (Fig. 8) are not as good as VAE on CIFAR10. I would also expect more ablation studies about how to pick p_{\had d}, which seems to be the key of this approach, in MNIST and CIFAR10. 
In terms of writing, the paper is a bit confusing in terms of motivations and notations. 
Overall, the method looks incremental and experimental results are mixed on small datasets so I vote for rejection. Note that I am not an expert on GAN/VAE so I put low confidence here.