Adversarial Multiple Source Domain Adaptation
While domain adaptation has been actively researched, most algorithms focus on the single-source-single-target adaptation setting. In this paper we propose new generalization bounds and algorithms under both classification and regression settings for unsupervised multiple source domain adaptation. Our theoretical analysis naturally leads to an efficient learning strategy using adversarial neural networks: we show how to interpret it as learning feature representations that are invariant to the multiple domain shifts while still being discriminative for the learning task. To this end, we propose multisource domain adversarial networks (MDAN) that approach domain adaptation by optimizing task-adaptive generalization bounds. To demonstrate the effectiveness of MDAN, we conduct extensive experiments showing superior adaptation performance on both classification and regression problems: sentiment analysis, digit classification, and vehicle counting.
Quality: The paper tackles the problem of domain adaptation. The main proposal of the paper is to find a multi-source generalization bound equivalent of a previously established single-source bound in [9]. Accordingly, two objective functions are devised to tighten the bound for multi-source domain adaptation. The experiments clearly shows that optimizing the proposed objective is more effective than naive extension of single-source objectives. The main paper is technically sound both the theories and the experiments seem valid to the eye of the reviewer.   Clarity: The paper is easy to follow. It motivates the problem, puts forward the formal notation in a concise and readable format and builds up the proposed approach on that.  The hyper-parameters necessary to reproduce the experimental results come in the appendix. The paper would strongly benefit from a thorough theoretical and empirical analysis of the convex coefficients \alpha on the tightness of the bound as well as the optimization. The reviewer appreciates the design of intuitive adaptive coefficients in 6 but more discussions and analysis would have been very helpful.  Originality: The paper builds on top of an existing work on single source domain adaptation and generalizes it to a special case of multi-source domain adaptation. The assumption about the setups of the multi-source domain adaptation (availability of mk unlabelled target data) is not too restrictive.  The related works section is limited, other works on multi-source domain adaptation exist that are not cited. For instance, Sun et al. NIPS 2011, or Choi et al. CVPR 2018.   Significance: - the paper puts forward a specific generalization of a previously published single-source domain adaptation error bound to multiple sources. The generalization is simple and intuitive.  - The paper tests that optimizing the derived bounds is better than naive approaches to multi-source domain adaptation. In that regard, it conclusively shows the benefits. However, it fails to compare to other approaches that are taken for multi-source domain adaptation (such as the ones mentioned above). - Another issue with the experiments is that when comparing C-DANN with the proposed MDAN it seems like C-DANN can achieve comparable results and on rare occasions even better results. This raises the need for baselines where more advanced domain adaptation techniques are used with combined source domains as a single domain.  ------------------- I read the rebuttal and the reviewers' comments. It seems I agree with all the reviewers' comments and the rebuttal makes sense. I will keep my rating and vouch for the acceptance of the paper.