ADVERSARIAL DROPOUT REGULARIZATION
We present a domain adaptation method for transferring neural representations from label-rich source domains to unlabeled target domains. Recent adversarial methods proposed for this task learn to align features across domains by "fooling" a special domain classifier network. However, a drawback of this approach is that the domain classifier simply labels the generated features as in-domain or not, without considering the boundaries between classes. This means that ambiguous target features can be generated near class boundaries, reducing target classification accuracy. We propose a novel approach, Adversarial Dropout Regularization (ADR), which encourages the generator to output more discriminative features for the target domain. Our key idea is to replace the traditional domain critic with a critic that detects non-discriminative features by using dropout on the classifier network. The generator then learns to avoid these areas of the feature space and thus creates better features. We apply our ADR approach to the problem of unsupervised domain adaptation for image classification and semantic segmentation tasks, and demonstrate significant improvements over the state of the art.
Unsupervised Domain adaptation is the problem of training a classifier without labels in some target domain if we have labeled data from a (hopefully) similar dataset with labels. For example, training a classifier using simulated rendered images with labels, to work on real images. 
Learning discriminative features for the target domain is a fundamental problem for unsupervised domain adaptation. The problem is challenging (and potentially ill-posed) when no labeled examples are given in the target domain. This paper proposes a new training technique called ADR, which tries to learn discriminative features for the target domain. The key idea of this technique is to move the target-domain features away from the source-domain decision boundary. ADR achieves this goal by encouraging the learned features to be robust to the dropout noise applied to the classifier.
My main concern about this paper is that the idea of "placing the target-domain features far away from the source-domain decision boundary" does not necessarily lead to *discriminative features* for the target domain. In fact, it is easy to come up with a counter-example: the target-domain features are far from the *source-domain* decision boundary, but they are all (both the positive and negative examples) on the same side of the boundary, which leads to poor target classification accuracy. The loss function (Equations 2-5) proposed in the paper does not prevent the occurrence of this counter-example.
Another concern comes from using the proposed idea in training a GAN (Section 4.3). Generating fake images that are far away from the boundary (as forced by the first term of Equation 9) is somewhat opposite to the objective of GAN training, which aims at aligning distributions of real and fake images. Although the second term of Equation 9 tries to make the generated and the real images similar, the paper does not explain how to properly balance the two terms of Equation 9. As a result, I am worried that the proposed method may lead to more mode-collapsing for GAN.
The experimental evaluation seems solid for domain adaptation. The semi-supervised GANs part seemed significantly less developed and might be weakening rather than strengthening the paper. 
Overall the performance of the proposed method is quite well done and the results are encouraging, despite the lack of theoretical foundations for this method.