A Unified Feature Disentangler for Multi-Domain Image Translation and Manipulation
We present a novel and unified deep learning framework which is capable of learning domain-invariant representation from data across multiple domains. Realized by adversarial training with additional ability to exploit domain-specific information, the proposed network is able to perform continuous cross-domain image translation and manipulation, and produces desirable output images accordingly. In addition, the resulting feature representation exhibits superior performance of unsupervised domain adaptation, which also verifies the effectiveness of the proposed model in learning disentangled features for describing cross-domain data.
Paper summary: The authors propose a VAE-GAN architecture for learning multi-domain image representations where the domain information is disentangled from the domain invariant information, while simultaneously generating images of all considered domains. A domain classifier D_v pushes the VAE encoder E to learn a domain-invariant embedding z by adversarial training. This embedding is then concatenated with a one-hot encoding v of the domain before being fed to the VAE decoder G, which is then able to reconstruct domain-appropriate images. The VAE is further paired with an adversarial real / fake classifier, which improves image sharpness, and a domain classifier, which promotes mutual information between the generated images and the domain encoding v. Experiments are performed on human face generation and semi-supervised digits classification.  Comments: The proposed approach is pretty straightforward when compared to recent works on image generation / feature disentanglement (which is good), and the "unified" nature of the proposed architecture must be appreciated. The paper is well written and easy to follow, although some technical details are not sufficiently clear from the text (point #2 below). Furthermore, in my opinion the authors tend to overstate their contributions w.r.t existing methods, in particular the one in [17] (point #1 below). Finally, the experimental evaluation on image generation is a bit lacking (point #3 below). My current vote is for rejection, but I'm willing to increase my score given satisfying answers to the following three main points:  1) In my opinion the novelty of the proposed method w.r.t. the one in [17] is quite limited. In fact, the proposed network can be directly derived from the E-CDRD on of [17] by simply merging the source and target branches and adding a domain adversarial classifier to the hidden representation. Can you please elaborate on this? Are there any other differences (theoretical or practical / architectural) that I'm missing? 2) I think that the overall clarity of the paper would benefit from moving part of the information reported in the supplementary material to the main paper. In particular, I find it a bit odd that nowhere in the paper is ever mentioned that v is a one-hot encoding of the domain labels, or that no information is ever given about the architectures of the various modules composing the network. 3) All the results shown on image generation are of a qualitative kind, and there's no comparison with existing methods. While, unfortunately, this is becoming alarmingly common in this line of research, I still think that authors should strive to evaluate as objectively as possible their proposed methods. In fact, it's not inconceivable that the generated images shown in the paper could be carefully selected outliers. Performing this kind of objective evaluation is a non-trivial tasks, but some de-facto standards currently exist, including Mechanical Turk-based perceptual evaluation (like in [3]), Inception score (like in REF1 below), or GAN convergence metrics. In the absence of any of these I would argue that, from the purely scientific point of view, the results in Section 4.2 are effectively meaningless.  REF1: Tim  Salimans,  Ian  Goodfellow,  Wojciech  Zaremba,  Vicki  Cheung,  Alec  Radford,  and  Xi  Chen. Improved techniques for training gans. In Neural Information Processing Systems (NIPS), 2016  --- Additional comments after author response phase --- The authors gave satisfactory answers to my three main points of criticism, although I still think that the novelty of the proposed method w.r.t the state of the art is a bit limited and should not be emphasized too much in the text. Overall, I think the paper can be accepted.