Bayesian Model-Agnostic Meta-Learning
Due to the inherent model uncertainty, learning to infer Bayesian posterior from a few-shot dataset is an important step towards robust meta-learning. In this paper, we propose a novel Bayesian model-agnostic meta-learning method. The proposed method combines efficient gradient-based meta-learning with nonparametric variational inference in a principled probabilistic framework. Unlike previous methods, during fast adaptation, the method is capable of learning complex uncertainty structure beyond a simple Gaussian approximation, and during meta-update, a novel Bayesian mechanism prevents meta-level overfitting. Remaining a gradientbased method, it is also the first Bayesian model-agnostic meta-learning method applicable to various tasks including reinforcement learning. Experiment results show the accuracy and robustness of the proposed method in sinusoidal regression, image classification, active learning, and reinforcement learning.
The paper introduces a Bayesian variant of the model-agnostic meta-learning (MAML) algorithm. MAML works by trying to find an "initial" parameter point from which one can adapt well with a few gradient update steps to different tasks. The first extension (Bayesian fast adaptation) generalizes this into an ensemble, where a set of particles is updated using Stein variational gradient descent to approximate task training posteriors and the validation loss that is used for the meta-update. BMAML extends this further such that the "validation" loss is changed to a loss between the (approximate) posteriors of the task-training and task-training-plus-validation distributions. MAML and BMAML (together with ensemble version of MAML) are compared in regression, classification, and reinforcement learning experiments.  Paper is mainly excellently written and the method is very interesting extension of MAML that seems to clearly improve the performance in the presented experiments. Yet, I have some confusions about the interpretation of the method.  Main comments: (1) The proposed method seems very much tied to a computational algorithm (SVGD) rather than be a model or procedure description in a Bayesian sense (where one doesn't usually need to specify specific computational algorithms, but models and/or utilities). Also, the used number of SVGD steps seems so small that it seems questionable if the distributions have converged such that they could be thought of as even approximately the posteriors. And if they would be ran to convergence, the initial particles would not have an effect anymore? This makes the meaning of, for example, the train-task posterior and it's conditioning on Theta_0 very vague. (Unless I misunderstood some parts of the method.) (2) The interpretation of the initial particles Theta_0 seems somewhat vague in the manuscript (see also comment 1). Can they be interpreted as representing an implicit prior (or is the implicit prior a combination of these and the SVGD instance, as Grant et al.'s interpretation seems to be based on a quick glance, but which point of view is not given in this paper)? (3) The BMAML meta-objective is trying to find a set of initial particles, such that the approx. posterior with task-train data is similar to the approx. posterior of the task-train-plus-validation data. Again from a Bayesian perspective, this sounds a bit weird that one tries to match posteriors where the only difference is that one has more data than the other (and the somewhat vague conditioning on Theta_0). On the other hand, a practical interpretation of the this is that one is trying to fit the initial particles such that adding the validation data to the training data doesn't change the parameter distribution much. (4) Wouldn't another way of trying to avoid the overfitting of Bayesian fast adaption be to use a proper scoring rule or some such loss function that would evaluate the full posterior predictive distribution and require honest uncertainties rather than a point prediction in Equation 5? (5) Why are there no comparisons to Grant et al. paper? Would it be too computationally costly for the experiments here? Also, why isn't the Bayesian fast adaptation method included in the results? It is mentioned that initial experiments found that it might overfit, but it would still be interesting to compare against to understand where the main improvement of BMAML comes from. (6) One mentioned motivation is that Gaussian approx. as in Grant et al. would not be appropriate in many cases (e.g., BNNs). Yet, the number of particles used is rather small and one would not expect it to be able to represent any complicated posteriors very well. (7) Will code be made available?  In summary, it seems that my main confusions arise from the interpretation of Theta_0 and the interpretation of the distributions, e.g., p(theta_tau | D^trn_tau, Theta_0), as posteriors while actually the conditioning on Theta_0 seems to require that the inference procedure is not run to convergence, so that they would actually represent the posteriors. Hopefully, the authors could clarify the interpretation of the method (or point out my misunderstandings). Apart from this, I think the method seems to be an interesting extension of MAML and would be a relevant development of this line of work for NIPS.  Update after author feedback: I appreciate the authors' careful response and hope that they revise the manuscript in line with it. I'm still a bit conflicted about the approach from a Bayesian point of view, as it explicitly relies on not running the inference procedure (SVGD) for too long (that is, to actual convergence), since then the effect of the initial set of parameters Theta_0, and hence the meta-learning, would be lost (at least in theory, although possibly not in practice fully). And I'm wondering whether the approach could be formulated in some more principled manner with regard to this issue (e.g., the more direct way of using a proper scoring rule; I'm not sure if I agree that this would be a less "principled Bayesian" way of formulating the problem). On the other hand, from a practical point of view, I think the approach is very interesting and, overall, I think the work is certainly worth publishing. And I guess actually the original MAML has much of this same flavour of as it relies on a limited number of gradient updates (but, of course, it's not and not advertised as a Bayesian method).