Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
Meta-learning allows an intelligent agent to leverage prior learning episodes as a basis for quickly improving performance on a novel task. Bayesian hierarchical modeling provides a theoretical framework for formalizing meta-learning as inference for a set of parameters that are shared across tasks. Here, we reformulate the model-agnostic meta-learning algorithm (MAML) of Finn et al. (2017) as a method for probabilistic inference in a hierarchical Bayesian model. In contrast to prior methods for meta-learning via hierarchical Bayes, MAML is naturally applicable to complex function approximators through its use of a scalable gradient descent procedure for posterior inference. Furthermore, the identification of MAML as hierarchical Bayes provides a way to understand the algorithm’s operation as a meta-learning procedure, as well as an opportunity to make use of computational strategies for efficient inference. We use this opportunity to propose an improvement to the MAML algorithm that makes use of techniques from approximate inference and curvature estimation.
 MAML (Finn+ 2017) is recast as a hierarchical Bayesian learning procedure. In particular the inner (task) training is initially cast as point-wise max likelihood estimation, and then (sec4) improved upon by making use of the Laplace approximation. Experimental evidence of the relevance of the method is provided on a toy task involving a NIW prior of Gaussians, and the (benchmark) MiniImageNet task. Casting MAML as HB seems a good idea. The paper does a good job of explaining the connection, but I think the presentation could be clarified. The role of the task prior and how it emerges from early stopping (ie a finite number of gradient descent steps) (sec 3.2) is original and technically non-trivial, and is a contribution of this paper. The synthetic data experiment sec5.1 and fig5 is clearly explained and serves to additionally clarify the proposed method. Regarding the MiniImageNet experiments, I read the exchange on TCML and agree with the authors of the paper under review. However, I recommend including the references to Mukhdalai 2017 and Sung 2017 in the footnote on TCML to strengthen the point more generically, and show that not just TCML but other non-shallow architectures are not considered for comparison here. In addition, the point made by the TCML authors is fair ("nothing prevented you from...") and I would also recommend mentioning the reviewed paper*s authors* decision (not to test deeper architectures) in the footnote. This decision is in order but needs to be stated in order for the reader to form a balanced view of methods at her disposal. The experimental performance reported Table 1 remains small and largely within one standard deviation of competitor methods. I am assessing this paper as "7" because despite the merit of the paper, the relevance of the reformulation of MAML, and the technical steps involved in the reformulation, the paper does not eg address other forms (than L-MAML) of the task-specific subroutine ML-..., and the benchmark improvements are quite small. I think the approach is good and fruitful. # Suggestions on readability * I have the feeling the paper inverts from their use in Finn 2017 (step size for meta- vs task-training). This is unfortunate and will certainly confuse readers; I advise carefully changing this throughout the entire paper (eg Algo 2,3,4, eq 1, last eq in sec3.1, eq in text below eq3, etc) * I advise avoiding the use of the symbol f, which appears in only two places in Algo 2 and the end of sec 3.1. This is in part because f is given another meaning in Finn 2017, but also out of general parsimony in symbol use. (could leave the output of ML-... implicit by writing ML-...(.theta, T)_j in the ; if absolutely needed, use another symbol than f) * Maybe sec3 can be clarified in its structure by re-ordering points on the quadratic error function and early stopping (eg avoiding to split them between end of 3.1 and 3.2). * sec6 "Machine learning and deep learning": I would definitely avoid this formulation, seems to tail in with all the media nonsense on "what*s the difference between ML and DL ?". In addition the formulation seems to contrast ML with hierarchical Bayesian modeling, which does not make sense/ is wrong and confusing. # Typos * sec1 second parag: did you really mean "in the architecture or loss function"? unclear. * sec2: over a family * "common structure, so that" (not such that) * orthgonal * sec2.1 suggestion: clarify that .theta and .phi are in the same space * sec2.2 suggestion: task-specific parameter is distinct from ... parameters .phi_j$ (I would avoid writing just .phi altogether to distinguish in usage from .theta) * Gaussian-noised * approximation of the it objective * before eq9: "that solves": well, it doesn*t really "solve" the minimisation, in that it is not a minimum; reformulate this? * sec4.1 innaccurate * well approximated * sec4.2 an curvature * (Amari 1989) * For the the Laplace * O(n^3) : what is n ? * sec5.2 (Ravi and L 2017) * for the the
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
Meta-learning allows an intelligent agent to leverage prior learning episodes as a basis for quickly improving performance on a novel task. Bayesian hierarchical modeling provides a theoretical framework for formalizing meta-learning as inference for a set of parameters that are shared across tasks. Here, we reformulate the model-agnostic meta-learning algorithm (MAML) of Finn et al. (2017) as a method for probabilistic inference in a hierarchical Bayesian model. In contrast to prior methods for meta-learning via hierarchical Bayes, MAML is naturally applicable to complex function approximators through its use of a scalable gradient descent procedure for posterior inference. Furthermore, the identification of MAML as hierarchical Bayes provides a way to understand the algorithm’s operation as a meta-learning procedure, as well as an opportunity to make use of computational strategies for efficient inference. We use this opportunity to propose an improvement to the MAML algorithm that makes use of techniques from approximate inference and curvature estimation.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 MAML (Finn+ 2017) is recast as a hierarchical Bayesian learning procedure. In particular the inner (task) training is initially cast as point-wise max likelihood estimation, and then (sec4) improved upon by making use of the Laplace approximation. Experimental evidence of the relevance of the method is provided on a toy task involving a NIW prior of Gaussians, and the (benchmark) MiniImageNet task. Casting MAML as HB seems a good idea. The paper does a good job of explaining the connection, but I think the presentation could be clarified. The role of the task prior and how it emerges from early stopping (ie a finite number of gradient descent steps) (sec 3.2) is original and technically non-trivial, and is a contribution of this paper. The synthetic data experiment sec5.1 and fig5 is clearly explained and serves to additionally clarify the proposed method. Regarding the MiniImageNet experiments, I read the exchange on TCML and agree with the authors of the paper under review. However, I recommend including the references to Mukhdalai 2017 and Sung 2017 in the footnote on TCML to strengthen the point more generically, and show that not just TCML but other non-shallow architectures are not considered for comparison here. In addition, the point made by the TCML authors is fair ("nothing prevented you from...") and I would also recommend mentioning the reviewed paper*s authors* decision (not to test deeper architectures) in the footnote. This decision is in order but needs to be stated in order for the reader to form a balanced view of methods at her disposal. The experimental performance reported Table 1 remains small and largely within one standard deviation of competitor methods. I am assessing this paper as "7" because despite the merit of the paper, the relevance of the reformulation of MAML, and the technical steps involved in the reformulation, the paper does not eg address other forms (than L-MAML) of the task-specific subroutine ML-..., and the benchmark improvements are quite small. I think the approach is good and fruitful. # Suggestions on readability * I have the feeling the paper inverts from their use in Finn 2017 (step size for meta- vs task-training). This is unfortunate and will certainly confuse readers; I advise carefully changing this throughout the entire paper (eg Algo 2,3,4, eq 1, last eq in sec3.1, eq in text below eq3, etc) * I advise avoiding the use of the symbol f, which appears in only two places in Algo 2 and the end of sec 3.1. This is in part because f is given another meaning in Finn 2017, but also out of general parsimony in symbol use. (could leave the output of ML-... implicit by writing ML-...(.theta, T)_j in the ; if absolutely needed, use another symbol than f) * Maybe sec3 can be clarified in its structure by re-ordering points on the quadratic error function and early stopping (eg avoiding to split them between end of 3.1 and 3.2). * sec6 "Machine learning and deep learning": I would definitely avoid this formulation, seems to tail in with all the media nonsense on "what*s the difference between ML and DL ?". In addition the formulation seems to contrast ML with hierarchical Bayesian modeling, which does not make sense/ is wrong and confusing. # Typos * sec1 second parag: did you really mean "in the architecture or loss function"? unclear. * sec2: over a family * "common structure, so that" (not such that) * orthgonal * sec2.1 suggestion: clarify that .theta and .phi are in the same space * sec2.2 suggestion: task-specific parameter is distinct from ... parameters .phi_j$ (I would avoid writing just .phi altogether to distinguish in usage from .theta) * Gaussian-noised * approximation of the it objective * before eq9: "that solves": well, it doesn*t really "solve" the minimisation, in that it is not a minimum; reformulate this? * sec4.1 innaccurate * well approximated * sec4.2 an curvature * (Amari 1989) * For the the Laplace * O(n^3) : what is n ? * sec5.2 (Ravi and L 2017) * for the the
assistant
 MAML (Finn+ 2017) is recast as a hierarchical Bayesian learning procedure. In particular the inner (task) training is initially cast as point-wise max likelihood estimation, and then (sec4) improved upon by making use of the Laplace approximation. Experimental evidence of the relevance of the method is provided on a toy task involving a NIW prior of Gaussians, and the (benchmark) MiniImageNet task. Casting MAML as HB seems a good idea. The paper does a good job of explaining the connection, but I think the presentation could be clarified. The role of the task prior and how it emerges from early stopping (ie a finite number of gradient descent steps) (sec 3.2) is original and technically non-trivial, and is a contribution of this paper. The synthetic data experiment sec5.1 and fig5 is clearly explained and serves to additionally clarify the proposed method. Regarding the MiniImageNet experiments, I read the exchange on TCML and agree with the authors of the paper under review. However, I recommend including the references to Mukhdalai 2017 and Sung 2017 in the footnote on TCML to strengthen the point more generically, and show that not just TCML but other non-shallow architectures are not considered for comparison here. In addition, the point made by the TCML authors is fair ("nothing prevented you from...") and I would also recommend mentioning the reviewed paper*s authors* decision (not to test deeper architectures) in the footnote. This decision is in order but needs to be stated in order for the reader to form a balanced view of methods at her disposal. The experimental performance reported Table 1 remains small and largely within one standard deviation of competitor methods. I am assessing this paper as "7" because despite the merit of the paper, the relevance of the reformulation of MAML, and the technical steps involved in the reformulation, the paper does not eg address other forms (than L-MAML) of the task-specific subroutine ML-..., and the benchmark improvements are quite small. I think the approach is good and fruitful. # Suggestions on readability * I have the feeling the paper inverts from their use in Finn 2017 (step size for meta- vs task-training). This is unfortunate and will certainly confuse readers; I advise carefully changing this throughout the entire paper (eg Algo 2,3,4, eq 1, last eq in sec3.1, eq in text below eq3, etc) * I advise avoiding the use of the symbol f, which appears in only two places in Algo 2 and the end of sec 3.1. This is in part because f is given another meaning in Finn 2017, but also out of general parsimony in symbol use. (could leave the output of ML-... implicit by writing ML-...(.theta, T)_j in the ; if absolutely needed, use another symbol than f) * Maybe sec3 can be clarified in its structure by re-ordering points on the quadratic error function and early stopping (eg avoiding to split them between end of 3.1 and 3.2). * sec6 "Machine learning and deep learning": I would definitely avoid this formulation, seems to tail in with all the media nonsense on "what*s the difference between ML and DL ? ". In addition the formulation seems to contrast ML with hierarchical Bayesian modeling, which does not make sense/ is wrong and confusing. # Typos * sec1 second parag: did you really mean "in the architecture or loss function"? unclear. * sec2: over a family * "common structure, so that" (not such that) * orthgonal * sec2.1 suggestion: clarify that .theta and .phi are in the same space * sec2.2 suggestion: task-specific parameter is distinct from ... parameters .phi_j$ (I would avoid writing just .phi altogether to distinguish in usage from .theta) * Gaussian-noised * approximation of the it objective * before eq9: "that solves": well, it doesn*t really "solve" the minimisation, in that it is not a minimum; reformulate this? * sec4.1 innaccurate * well approximated * sec4.2 an curvature * (Amari 1989) * For the the Laplace * O(n^3) : what is n ? * sec5.2 (Ravi and L 2017) * for the the