Gradient-based meta-learners such as MAML (Finn et al., 2017) are able to learn a meta-prior from similar tasks to adapt to novel tasks from the same distribution with few gradient updates. One important limitation of such frameworks is that they seek a common initialization shared across the entire task distribution, substantially limiting the diversity of the task distributions that they are able to learn from. In this paper, we augment MAML with the capability to identify tasks sampled from a multimodal task distribution and adapt quickly through gradient updates. Specifically, we propose a multimodal MAML algorithm that is able to modulate its meta-learned prior according to the identified task, allowing faster adaptation. We evaluate the proposed model on a diverse set of problems including regression, few-shot image classification, and reinforcement learning. The results demonstrate the effectiveness of our model in modulating the meta-learned prior in response to the characteristics of tasks sampled from a multimodal distribution.
Strengths:
+ The paper identifies a valid limitation of the MAML algorithm: With a limited number of gradient descent steps from a single initialization, there is a limit to the ability of a fixed-size neural network to adapt to tasks sampled from a diverse dataset.
+ The tSNE plots show some preliminary interesting structure for the simple regression and RL tasks, but not for the classification task.
Weaknesses:
- The motivation of uncovering latent modes of a task distribution does not align with the proposed method. The algorithm computes a continuous representation of the data from a task (which is fixed during gradient-based fast adaptation). The mode identity, on the other hand, should be a discrete variable.  Such a discrete variable is never explicitly computed in the proposed method.
- The technical writing is unclear and jargon is often used without definition. Importantly, one of the central motivators of the paper, "task modulation", is never given a precise definition.
- The standard few-shot classification task (Omniglot) does not clearly consist of a task distribution that is multimodal, so the method is not well-motivated in this setting.
- Experimental conclusions are weak.
Major comments:
- The paper neglects to discuss how the proposed method could be used in the context of other methods for "gradient-based meta-learning" such as Ravi & Larochelle (2016). I believe the attention-based modulation and the FiLM modulation could be easily adapted to that setting. Why was this not discussed or evaluated?
- Conditioning has been used in the context of few-shot learning before, but this is not discussed (https://arxiv.org/abs/1805.10123, https://arxiv.org/abs/1806.07528).
- The paper often confounds task representation with neural network parameter values. For example, Figure 1 depicts the adaptation of parameter values with gradients (\nabla L), yet the caption describes "task modes." More careful writing would disentangle these two components.
- The motivation for the particular form of the task embedding computation is not given. What were the other options? Why not, for example, an order-invariant function instead of a bidirectional GRU?
- In all of the experiments, there is no appropriate baseline that keeps the parameter dimensionality constant, so it is unclear whether the (marginal) improvement in performance is due to added expressivity by adding more parameters rather than an algorithmic improvement. I suggest an ensembling baseline with an appropriate number of ensemble members.
- There is no evaluation on a standard benchmark for few-shot classification (miniImageNet), and the Omniglot improvement is small.
- The reinforcement learning comparison at some point compares MUMOMAML with modulation applied (therefore with access to task-specific data) to MAML with no adaptation (and therefore no access to task-specific data). This is not entirely fair.
- tSNE results can be misleading (e.g., see https://distill.pub/2016/misread-tsne/), and the task delineation is not extremely clean. I would be more convinced if a clustering algorithm were applied.
Minor comments:
- The paper needs to be checked over for English grammar and style.
- everywhere: The "prior" referred to in this paper is not a prior in the Bayesian sense. I suggest a more careful use of terminology.
- abstract: "augment existing gradient-based meta-learners" You augment a specific variant of gradient-based meta-learning, MAML.
- pg. 1: "carve on a snowboard" don't know what this means
- The terminology of "task distribution" and "modes" thereof is used without introduction in the introduction section. The terminology "model-based meta-learning/adaptation" and "gradient-based meta-learning/adaptation" is also used without introduction here. This makes the introduction unnecessarily opaque. Consider the reader who is not familiar with meta-learning papers; they would have a very hard time parsing, for example, the phrase "...this not only requires additional identity information about the modes, which is not always available or is ambiguous when the modes are not clearly disjoint..." (pg. 1).
- Further, the terminology "model-based" seems non-standard, and is aliased with the term model-based reinforcement learning (which specifically refers to the set of RL algorithms that make use of a "model" of transition dynamics). Since the paper tackles a reinforcement learning benchmark, this may lead to some confusion.
- pg. 3; "our model does not maintain an internal state" Is the task representation/embedding not an internal state?
- pg. 3: "relevant but vaguely related skills" this is imprecise
- pg. 3: The episodic training setup, which is standard to meta-learning setups, could be much better described. The MAML algorithm could be given better intuition.
- everywhere: "task specific" -> task-specific
- Algorithm 1: "infer" is a misuse of terminology that usually refers to an operation in latent variable probabilistic modelling. Since the computation of \tau is purely feedforward, I recommend writing "compute."
- \tau should be used in some places where v is used instead