In unsupervised domain adaptation (UDA), classifiers for the target domain (TD) are trained with clean labeled data from the source domain (SD) and unlabeled data from TD. However, in the wild, it is hard to acquire a large amount of perfectly clean labeled data in SD given limited budget. Hence, we consider a new, more realistic and more challenging problem setting, where classifiers have to be trained with noisy labeled data from SD and unlabeled data from TD-we name it wildly UDA (WUDA). We show that WUDA ruins all UDA methods if taking no care of label noise in SD, and to this end, we propose a Butterfly framework, a powerful and efficient solution to WUDA. Butterfly maintains four models (e.g., deep networks) simultaneously, where two take care of all adaptations (i.e., noisy-to-clean, labeled-to-unlabeled, and SD-to-TD-distributional) and then the other two can focus on classification in TD. As a consequence, Butterfly possesses all the conceptually necessary components for solving WUDA. Experiments demonstrate that under WUDA, Butterfly significantly outperforms existing baseline methods.
This paper introduces the idea of wildly unsupervised domain adaptation, where the source labels are noisy and the target data is unsupervised. To tackle this, the authors propose an architecture based one two branches: one acting on the mixed source-target data and the other on the target data only. During training, each branch is updated using the idea of co-teaching, by finding the samples with the lowest loss values. Pseudo-labeling is then applied to the target data, and the process iterates.
Originality:
- In essence, the proposed method combines co-teaching (Han et al., 2018) and pseudo-labeling (Saito et al., 2017). While it goes beyond the naive two-stage approach, used here as a baseline, the technical novelty remains limited.
- The main novelty consists of using two branches to model the two domains. However, the necessity for the second branch is not very clearly explained, and remains obscure to me, since the first branch already acts on the mixed source-target data.
Clarity:
In addition the fact that, as mentioned above, the design of the overall framework is not entirely well motivated, I found the paper somewhat hard to follow. In particular:
- One has to go to the appendix to find Alg. 2, which describes one of the key components (the Checking method in Alg. 1). The steps performed by Alg. 2 are not explained anywhere.
- In Alg. 1, it seems surprising that \tilde{D}_T^l is initialized as \tilde{D}_s, since, according to Fig. 3, the second branch should act on the target data only.
- In Alg. 1, the authors do not explain how they initialize the values R(T) and R_t(T).
- I would expect that, to obtain meaningful results from the Checking method, the parameters of the networks F_1, F_2, F_{t1} and F_{t2} should already be initialized to reasonable values. Can the authors comment on the initialization procedure?
- In Alg. 2, how are the inner minimization problems solved? Are u_1 and u_2 truly enforced to be binary variables? How fast can one obtain the solutions to these problems?
- In Fig. 1, it is not clear to me what the term Interaction between the two branches represent. From the text, I could not find any reference to explicit interactions between the branches.
- In Eq. 1, the loss function \ell() is not defined (although I imagine that it is the cross-entropy).
Experiments:
- The experiments show the good behavior of the method. However, while I understand the motivation behind defining the two-stage baseline using ATDA, which is used in the proposed method, there seem to be no strict constrain on using this specific method in the two-stage scenario. For example, based on the results in Table 1, one might rather want to use TCL as the second stage, i.e., have a baseline Co+TCL. 
- As I mentioned before, the motivation behind the second branch in the framework is not clear to me. I would appreciate it if the authors could explain the reasoning behind this branch and evaluate their method without it.
Summary:
The novelty of the proposed method, relying on a combination of co-teaching and pseudo-labeling, is limited. Furthermore, the clarity of the paper could be significantly improved.