AMORTIZED BAYESIAN META-LEARNING
Meta-learning, or learning-to-learn, has proven to be a successful strategy in attacking problems in supervised learning and reinforcement learning that involve small amounts of data. State-of-the-art solutions involve learning an initialization and/or optimization algorithm using a set of training episodes so that the metalearner can generalize to an evaluation episode quickly. These methods perform well but often lack good quantification of uncertainty, which can be vital to realworld applications when data is lacking. We propose a meta-learning method which efficiently amortizes hierarchical variational inference across tasks, learning a prior distribution over neural network weights so that a few steps of Bayes by Backprop will produce a good task-specific approximate posterior. We show that our method produces good uncertainty estimates on contextual bandit and few-shot learning benchmarks.
This work proposes an adaptation to MAML-type models that accounts for posterior uncertainty in task specific latent variables. This is achieved via a hierarchical Bayesian view of MAML, employing variational inference for the task-specific parameters. The key intuition of this paper is that one can perform fast and efficient test-time variational inference for the task-specific latent variables by learning a good initialization during meta-training. This is achieved in a very similar fashion to MAML, and allows for an interesting form of amortization of test-time inference.
Pros:
- For the most part, the approach presented is principled and well justified.
- The motivation is clear: in the few-shot learning regime we expect to have little data to infer the task-specific latent variables, and so we should perform posterior inference to account for uncertainty.
- The paper is well written, clear, and easy to follow.
Cons (more details below):
- It is not clear what the significant contributions of this paper are, as a number of methods have been proposed to account for uncertainty in the task-specific latent variables, and results for many of these methods appear to be better than those presented here.
- Experimental section does compare to many of the existing related methods 
- There are some conceptual issues that need to be addressed by the authors. 
I enjoyed reading this paper, and I think the ideas and work presented are, for the most part, solid. However, I am not sure to what extent the novel contribution of this paper is significant. Several papers, including Grant et al. (2018), but going back to Heskes (2000), have proposed the hierarchical Bayesian view of meta-learning. Grant et al. (2018) used a Laplace approximation to learn in such a model with MAML-type settings, presenting a method that accounts for uncertainty in this family of models. More recently, Finn et al. (2018) and Kim et al. (2018) have done this in a variational manner, albeit with variations in the implementation details. Gordon et al. (2018) proposed a more general presentation, unifying the above works (and others) in a Bayesian framework that allows for different functional forms of posterior inference (both point estimates and distributional) of the task-specific parameters, including gradient based procedures. All of these papers have been publicly available for a few months at the time of submission, such that this view of meta-learning as (amortized) Bayesian inference is not novel.
Here are some points that I would ask the authors to address during the rebuttal period:
- The method presented in the paper does not account for the meta-training splits into query and test sets, other than to mention that these led to empirical performance gains (this is somewhat typical of probabilistic meta-learning papers). However, it not clear that this is justified from a probabilistic inference perspective, which would favour conditioning on all available data at inference time. Further, in the experimental section, the authors state that "For the few-shot learning experiments, we found it necessary to downweight the inner KL term for better performance in our model". Put together, it is not quite clear exactly what form of approximate inference is being conducted here. Can the authors comment on this?
- I am not sure I agree with the authors' interpretation of the term "amortized Bayesian inference", at least in that it deviates from the way the term is typically used in the related literature. The method negates the need to maintain variational parameters for each latent variable, and approximate posterior inference for unseen tasks may be performed relatively efficiently, which is highly desirable. However, a gradient optimization procedure must still be performed for inference of task-specific variables for new tasks at test time. Thus, new variational parameters must be introduced and optimized at test time. It is true that by finding good global initializations the authors may drastically reduce the computational cost of the inference process, but this implies that the cost of inference at test time has been reduced, not fully amortized to a fixed cost (unless one fixes the number of gradient steps, which is a further deviation from variational inference and requires a prior of the form used in Grant et al. (2018)). Full amortization of inference for the task-specific variables is proposed by Garnelo et al. (2018) and Gordon et al. (2018), as well as Edwards and Storkey (2016), all of which employ inference networks mapping directly from the query sets to the variational parameters of the latent variables. In these cases, posterior inference of the latent variables for unseen tasks has the constant cost of a pass through an inference network, rather than several forward-backward passes, and does not require introducing new variational parameters to be optimized. Further, these methods negate the need for differentiating through gradient-based procedures at meta-training time, which is not avoided in this paper, but rather dealt with in the standard Hessian-vector product form. It would be highly useful in the paper (perhaps in the related work section) for the authors to conduct a more thorough comparison of their proposed method and the existing literature employing amortized inference for meta-learning, to put their work in context.
I also have a number of concerns regarding the experimental section of the paper, which I find to be lacking both in details and the empirical comparison of the method to existing works.
- The authors' cite recent works on meta-learning that take into account uncertainty in the local latent variables (e.g., Grant et al. (2018), Finn et al. (2018), Kim et al. (2018)), but do not compare to these methods.
- Results from Garnelo et al. (2018) are not provided for the contextual bandits experiment. Their results seem to be comparable or better to those presented in this paper. Can the authors comment on this?
- The same is true for the few-shot learning case, where MAML is the only method compared to, despite there being, at the time of submission, many papers which have significantly improved upon these results.
- In terms of details, it is unclear how many gradient steps were taken at test time, and how this affects performance of the model.
- In terms of accuracy, the proposed method appears to be under-performing significantly (i.e., below confidence bounds in almost all cases).
- The statement "...we believe improvements could be made with better variance reduction methods for stochastic gradients" should, in my opinion, either be investigated or omitted from the paper.
- In terms of uncertainty quantification, I find this experimental evaluation highly interesting. However, there is not a comparison to much of the existing work. The comparison to MAML is only of moderate interest in this case, as MAML is a deterministic method and is not expected to perform well in this regard. A comparison to Probabilistic or Bayesian MAML (at the least) would be more convincing if uncertainty calibration proved to be better for this method.
Overall, the paper proposes a principled approach to performing approximate posterior inference for task specific latent variables in meta-learning settings. The paper is well-written, and the method is clearly derived. However, it is my impression that the paper does not make significant novel contributions to the existing research in (probabilistic) meta-learning, does not properly acknowledge all existing work (much of which covers the main ideas presented in the paper), has a number of conceptual issues that might need addressing, and its experimental section lacks evaluation and comparisons to the existing similar works. As the method is, for the most part, principled and well-derived, and the paper well written, I am willing to reconsider my overall score if the authors can demonstrate either (i) significant novelty or (ii) that this particular flavour of inference for the task-specific parameters provides significant benefits over existing approaches.
[1] - T. Heskes. Empirical Bayes for learning to learn. 2000.
[2] - E. Grant, C. Finn, S. Levine, T. Darrell, and T. Griffiths. Recasting gradient-based meta-learning as hierarchical Bayes. 2018.
[3] - C. Finn, K. Xu, and S. Levine. Probabilistic model-agnostic meta-learning. 2018.
[4] - T. Kim, J. Yoon, O. Dia, S. Kim, Y. Bengio, and S. Ahn. Bayesian model-agnostic meta-learning. 2018.
[5] - J. Gordon, J. Bronskill, M. Bauer, S. Nowozin, and R. Turner. Decision-theoretic meta-learning: versatile and efficient amortization of few-shot learning. 2018.
[6] - M. Garnelo, J. Schwarz, D. Rosenbaum, F. Viola, D. J. Rezende, S. Eslami, and Y. W. Teh. Neural processes. 2018.
[7] - H. Edwards, and A. Storkey. Towards a neural statistician. 2016.