MetaReg: Towards Domain Generalization using Meta-Regularization
Training models that generalize to new domains at test time is a problem of fundamental importance in machine learning. In this work, we encode this notion of domain generalization using a novel regularization function. We pose the problem of finding such a regularization function in a Learning to Learn (or) metalearning framework. The objective of domain generalization is explicitly modeled by learning a regularizer that makes the model trained on one domain to perform well on another domain. Experimental validations on computer vision and natural language datasets indicate that our method can learn regularizers that achieve good cross-domain generalization.
** Post Rebuttal ** The rebuttal adequately addresses all of my suggestions and questions. All of the new experiments in the rebuttal are valuable and should be included in the final. I recommend accepting the paper, assuming that the authors will add their new results to the paper.  ** Original Review ** This paper proposes a meta-learning approach to domain generalization, where the method learns a regularization term that is added to the task learning objective. Once the regularizer has been learned across different domains, the paper proposes to train on all domains using this regularizer. The paper additionally proposes to have a feature network that is shared across domains/tasks to enable more scalable meta-learning. The method provides a small, but significant improvement over simply training a model on all tasks, as well as prior domain generalization methods. The experiments provide additional ablations and visualizations for better understanding the method.  Pros: The paper addresses an important problem of the ability to generalize well to new domains given no data in the new domain, and provides an innovative approach based on learning to regularize to the learning process. The idea of learning a regularizer is interesting and relevant beyond the particular domain generalization problem setting addressed in this paper, increasing the potential impact of the paper. The experiments and analysis are thorough.  Cons: The empirical benefit provided compared to baseline models is relatively small. The clarity of the paper could be improved, both in terms of notation and how it is presented. Further, the Bayesian interpretation should be further fleshed out for it to be meaningful. There are details of the algorithm that are not included in the current version that would make it difficult to reproduce. [Detailed feedback below]  *More detailed comments & feedback* Related work: The idea of sharing a feature network across domains has been recently proposed in the meta-learning literature [2]. It would be good to reference [2] in the paper, for the sake of keeping track of relevant work. But, I do not penalize the authors because [2] is not published and came out ~3 months before the deadline. The related work has a detailed discussion of how this method relates to MLDG, but does not discuss the relation to other works. It would be helpful to include such discussion rather than simply summarizing prior methods.  Bayesian interpretation: The Bayesian interpretation doesn't completely make sense. First, what is the graphical model underlying this model? It would be helpful to have a Figure that illustrates the graphical model. Is it the same as the graphical model in [1]? [If so, it might be nice to reconcile notation with that prior work, with theta as the prior parameters and phi as the task parameters.] Second, why does a point estimate for phi make sense? How does the computation of this point estimate relate to the Bayesian interpretation? Prior work [1] developed a Bayesian interpretation of MAML using a MAP estimate to approximate an integral, drawing upon a previous connection between gradient descent and MAP inference. A MAP estimate provides a crude, but somewhat sensible approximation, given that the point has high probability. But, in this case, no such reasoning seems to exist. Discussing the connection to [1] would also be useful.  Comments on Clarity: 1. An algorithm for what happens at meta-test time would be really helpful. How are the task parameters theta_g initialized? (randomly or using the existing theta_i's?). How many gradient steps are taken at test time? Is it the same as l? Is both psi and theta updated, or just theta? 2. It would be helpful to have a diagram that illustrates the entire meta-learning approach. Figure 1b is a step in the right direction, but it would be nice to visualize the update to phi and to include multiple domains/thetas. The caption of Figure 1 is thorough, which is very nice. But, it doesn't describe how phi is used.  Algorithmic+Experimental suggestions and questions: 1. Right now, pairs of tasks are selected for training the regularizer. Since, at meta-test time, the model will be trained with the regularizer across multiple domains, it seems like it might make more sense to have meta-training of phi mimic this setting more closely, having each theta_i being trained across a set of domains, e.g. all but one domain, rather than an individual domain, and then training phi on domains not in that set. Right now, phi is trained only for thetas that were trained on an individual domain rather than multiple domains. 2. Would it make sense to have a replay memory of previous thetas? It seems like phi might learn to be a good regularizer for parameters that have been trained on a task for a long time (and might forget to be a good at regularizing randomly initialized parameters). 3. What are the implications of freezing the feature network? An ablation with varying amounts of layers in the feature network vs. task network could help study this question empirically. 4. In the experiments, it would also be useful to plot phi_i. 5. Are there experimental settings where we would expect more improvement? Perhaps in settings with more domains?  More minor comments: - a more common notation for a set of parameter vectors is to use uppercase \Theta (vs. \tilde{\theta}). - A lot of notation is definied in section 3.1 and some of it is not used later, such as M, T, \tilde{\theta}, \mathcal{D}. It might be better to consolidate some of this notation. - "generalization to a new domain is not guaranteed. To accomplish this, we propose...": This wording suggests that the proposed approach guarantees generalization to new domains, but theoretical guarantees are not discussed. - The last paragraph of section 3.1 is confusing because it isn't clear if there is one theta or multiple theta_i. When referring to theta, does the paper mean {theta_1, ..., theta_p} or an individual theta_i or theta_g? - In equation 4 and elsewhere, it would be helpful to put brackets around everything that is inside the gradient operator, to make it clear that the regularizer term is inside the gradient operator.  [1] Grant et al. ICLR 2018 [2] Zhou et al. http://arxiv.org/abs/1802.03596