Motivated by practical applications, we consider unsupervised domain adaptation for classification problems, in the presence of missing data in the target domain. More precisely, we focus on the case where there is a domain shift between source and target domains, while some components in the target data are systematically absent. We propose a way to impute non-stochastic missing data for a classification task by leveraging supervision from a complete source domain through domain adaptation. We introduce a single model performing joint domain adaptation, imputation and classification which is shown to perform well under various representative divergence families (H-divergence, Optimal Transport). We perform experiments on two families of datasets: a classical digit classification benchmark commonly used in domain adaptation papers and real world digital advertising datasets, on which we evaluate our model's classification performance in an unsupervised setting. We analyze its behavior showing the benefit of explicitly imputing non-stochastic missing data jointly with domain adaptation.
*Summary.* The paper presents and addresses the problem of performing domain adaptation when the target domain is systematically (i.e., not the result of a stochastic process) missing subsets of the data. The issue is motivated by applications where one modality of data becomes unavailable in the target domain (e.g., when deciding which ads to serve to new users, the predictor may have access to behavior across other websites but not on a specific merchant's website). The proposed method learns to map source and target data to a latent space where the representations for the source and target are aligned, the missing components of the target can be inferred, and classification can be performed successfully. These are achieved by adversarial/optimal transport loss on source and target features, a mean-squared error and adversarial loss on latent generation/imputation, and a cross entropy loss on source label prediction, respectively. Experiments are performed on digits and click-through rate (CTR) prediction and include a thorough set of baselines/oracles for comparison.
*Review.* While the problem statement is novel, I am unconvinced that the advertising experiment includes both a domain adaptation and imputation problem. I describe this in detail below. For this reason, I am giving the paper a weak reject.
*Questions that impacted rating.*
1. Ads experiment: From my understanding, the source domain is the traffic of users who have interacted with (clicked through to?) a specific partner and the target domain is the traffic of the users who have not interacted with that specific partner. The data that needs to be imputed is the click through rate for target users with that specific partner. In this case, it is not obvious to me why there is a domain shift between these two groups of users. This would imply that the traffic of source users and target users is different for other partners. I don't see why this would need to be true. Could the authors provide an explanation as to why this is the case (e.g., by showing that CTRs differ with other (partner, publisher) pairs between source and target). From my understanding, Table 5 only shows CTR averaged across all users in each domain, but does not show that the CTRs differ between source and target users for contexts/(partner, publisher) pairs (i.e., the results in table 5 could be due to the fact that the prior distribution over context is different for source and target users).
*Additional notes. Immaterial to rating.*
1. I personally felt that the motivation for UDA vs imputation in the first paragraph was a bit muddled. I think sticking to one example would make the motivation more clear to the reader. E.g., explain the prediction problem for medical imaging (which I assume is disease diagnosis, but it is not stated explicitly), describe how some medical imaging may be missing for certain patients (imputation), then explain that there may be noise across different medical imaging systems (UDA), then list the other applications where this arises with citations (e.g., These phenomena have also been documented in advertising applications [1], ...).
2. I was surprised by the difference between Adaptation-Partial and the other two train/test conditions in Figure 2 when p=30%. Out of curiosity, do the authors have an explanation for this discrepancy? I would have predicted that, if most of the information necessary for prediction was available in the remaining 70% of the image that the performance of these cases would be very similar.  I think it would be helpful to see the accuracy on the source domain and the labeled target domain to better understand that result.