Flexible Transfer Learning under Support and Model Shift
Transfer learning algorithms are used when one has sufficient training data for one supervised learning task (the source/training domain) but only very limited training data for a second task (the target/test domain) that is similar but not identical to the first. Previous work on transfer learning has focused on relatively restricted settings, where specific parts of the model are considered to be carried over between tasks. Recent work on covariate shift focuses on matching the marginal distributions on observations X across domains. Similarly, work on target/conditional shift focuses on matching marginal distributions on labels Y and adjusting conditional distributions P (X|Y ), such that P (X) can be matched across domains. However, covariate shift assumes that the support of test P (X) is contained in the support of training P (X), i.e., the training set is richer than the test set. Target/conditional shift makes a similar assumption for P (Y ). Moreover, not much work on transfer learning has considered the case when a few labels in the test domain are available. Also little work has been done when all marginal and conditional distributions are allowed to change while the changes are smooth. In this paper, we consider a general case where both the support and the model change across domains. We transform both X and Y by a location-scale shift to achieve transfer between tasks. Since we allow more flexible transformations, the proposed method yields better results on both synthetic data and real-world data.
This paper proposed a method to allow both the marginal and conditional distributions to change, which is less studied in the literature of transfer learning.In general, the paper is well written. But one thing is a bit vague for more general audience, i.e., why smoothness should be imposed and is so important in the proposed method. The authors should provide more details in the paper.This paper is motivated from [16] and [12], which assume the test X and Y can be transformed into new representations where the statistical property is somehow matched with corresponding domains. Though a bit incremental, the proposed method addressed an interesting transfer learning problem, and the experiments are sufficiently done to show the effectiveness of the proposed method.Minor issues:1. "mean-var-centered" should be "mean-var-scaled" in Figures 4 and 5.