Meta-Learning Probabilistic Inference for Prediction
This paper introduces a new framework for data efficient and versatile learning. Specifically: 1) We develop ML-PIP, a general framework for Meta-Learning approximate Probabilistic Inference for Prediction. ML-PIP extends existing probabilistic interpretations of meta-learning to cover a broad class of methods. 2) We introduce Versa{}, an instance of the framework employing a flexible and versatile amortization network that takes few-shot learning datasets as inputs, with arbitrary numbers of shots, and outputs a distribution over task-specific parameters in a single forward pass. Versa{} substitutes optimization at test time with forward passes through inference networks, amortizing the cost of inference and relieving the need for second derivatives during training. 3) We evaluate Versa{} on benchmark datasets where the method sets new state-of-the-art results, and can handle arbitrary number of shots, and for classification, arbitrary numbers of classes at train and test time. The power of the approach is then demonstrated through a challenging few-shot ShapeNet view reconstruction task.
 This paper proposes both a general meta-learning framework with approximate probabilistic inference, and implements an instance of it for few-shot learning. First, they propose Meta-Learning Probabilistic inference for Prediction (ML-PIP) which trains the meta-learner to minimize the KL-divergence between the approximate predictive distribution generated from it and predictive distribution for each class. Then, they use this framework to implement Versatile Amortized Inference, which they call VERSA. VERSA replaces the optimization for test time with efficient posterior inference, by generating distribution over task-specific parameters in a single forward pass. The authors validate VERSA against amortized and non-amortized variational inference which it outperforms. VERSA is also highly versatile as it can be trained with varying number of classes and shots. Pros - The proposed general meta-learning framework that aims to learn the meta-learner that approximates the predictive distribution over multiple tasks is quite novel and makes sense. - VERSA obtains impressive performance on both benchmark datasets for few-shot learning and is versatile in terms of number of classes and shots. - The appendix section has in-depth analysis and additional experimental results which are quite helpful in understanding the paper. Cons - The main paper feels quite empty, especially the experimental validation parts with limited number of baselines. It would have been good if some of the experiments could be moved into the main paper. Some experimental results such as Figure 4 on versatility does not add much insight to the main story and could be moved to appendix. - It would have been good if there was some validation of the time-performance of the model as one motivation of meta-learning is rapid adaptation to a test-time task. In sum, since the proposed meta-learning probabilistic inference framework is novel and effective I vote for accepting the paper. However the structure and organization of the paper could be improved by moving some of the methodological details and experimental results in the appendix to the main paper.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Meta-Learning Probabilistic Inference for Prediction
This paper introduces a new framework for data efficient and versatile learning. Specifically: 1) We develop ML-PIP, a general framework for Meta-Learning approximate Probabilistic Inference for Prediction. ML-PIP extends existing probabilistic interpretations of meta-learning to cover a broad class of methods. 2) We introduce Versa{}, an instance of the framework employing a flexible and versatile amortization network that takes few-shot learning datasets as inputs, with arbitrary numbers of shots, and outputs a distribution over task-specific parameters in a single forward pass. Versa{} substitutes optimization at test time with forward passes through inference networks, amortizing the cost of inference and relieving the need for second derivatives during training. 3) We evaluate Versa{} on benchmark datasets where the method sets new state-of-the-art results, and can handle arbitrary number of shots, and for classification, arbitrary numbers of classes at train and test time. The power of the approach is then demonstrated through a challenging few-shot ShapeNet view reconstruction task.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 This paper proposes both a general meta-learning framework with approximate probabilistic inference, and implements an instance of it for few-shot learning. First, they propose Meta-Learning Probabilistic inference for Prediction (ML-PIP) which trains the meta-learner to minimize the KL-divergence between the approximate predictive distribution generated from it and predictive distribution for each class. Then, they use this framework to implement Versatile Amortized Inference, which they call VERSA. VERSA replaces the optimization for test time with efficient posterior inference, by generating distribution over task-specific parameters in a single forward pass. The authors validate VERSA against amortized and non-amortized variational inference which it outperforms. VERSA is also highly versatile as it can be trained with varying number of classes and shots. Pros - The proposed general meta-learning framework that aims to learn the meta-learner that approximates the predictive distribution over multiple tasks is quite novel and makes sense. - VERSA obtains impressive performance on both benchmark datasets for few-shot learning and is versatile in terms of number of classes and shots. - The appendix section has in-depth analysis and additional experimental results which are quite helpful in understanding the paper. Cons - The main paper feels quite empty, especially the experimental validation parts with limited number of baselines. It would have been good if some of the experiments could be moved into the main paper. Some experimental results such as Figure 4 on versatility does not add much insight to the main story and could be moved to appendix. - It would have been good if there was some validation of the time-performance of the model as one motivation of meta-learning is rapid adaptation to a test-time task. In sum, since the proposed meta-learning probabilistic inference framework is novel and effective I vote for accepting the paper. However the structure and organization of the paper could be improved by moving some of the methodological details and experimental results in the appendix to the main paper.
assistant
 This paper proposes both a general meta-learning framework with approximate probabilistic inference, and implements an instance of it for few-shot learning. First, they propose Meta-Learning Probabilistic inference for Prediction (ML-PIP) which trains the meta-learner to minimize the KL-divergence between the approximate predictive distribution generated from it and predictive distribution for each class. Then, they use this framework to implement Versatile Amortized Inference, which they call VERSA. VERSA replaces the optimization for test time with efficient posterior inference, by generating distribution over task-specific parameters in a single forward pass. The authors validate VERSA against amortized and non-amortized variational inference which it outperforms. VERSA is also highly versatile as it can be trained with varying number of classes and shots. Pros - The proposed general meta-learning framework that aims to learn the meta-learner that approximates the predictive distribution over multiple tasks is quite novel and makes sense. - VERSA obtains impressive performance on both benchmark datasets for few-shot learning and is versatile in terms of number of classes and shots. - The appendix section has in-depth analysis and additional experimental results which are quite helpful in understanding the paper. Cons - The main paper feels quite empty, especially the experimental validation parts with limited number of baselines. It would have been good if some of the experiments could be moved into the main paper. Some experimental results such as Figure 4 on versatility does not add much insight to the main story and could be moved to appendix. - It would have been good if there was some validation of the time-performance of the model as one motivation of meta-learning is rapid adaptation to a test-time task. In sum, since the proposed meta-learning probabilistic inference framework is novel and effective I vote for accepting the paper. However the structure and organization of the paper could be improved by moving some of the methodological details and experimental results in the appendix to the main paper.