main.pdf
We propose CAML, a meta-learning method for fast adaptation that partitions the model parameters into two parts: context parameters that serve as additional input to the model and are adapted on individual tasks, and shared parameters that are meta-trained and shared across tasks. At test time, the context parameters are updated with one or several gradient steps on a task-specific loss that is backpropagated through the shared part of the network. Compared to approaches that adjust all parameters on a new task (e.g., MAML), our method can be scaled up to larger networks without overfitting on a single task, is easier to implement, and saves memory writes during training and network communication at test time for distributed machine learning systems. We show empirically that this approach outperforms MAML, is less sensitive to the task-specific learning rate, can capture meaningful task embeddings with the context parameters, and outperforms alternative partitionings of the parameter vectors.
Summary
CAML is a gradient-based meta-learning method closely related to MAML. It divides model parameters into disjoint sets of task-specific parameters $\phi$ which are adapted to each task and task-independent parameters $\theta$ with are meta-learned across tasks. $\phi$ are then interpreted as an embedding and fed as input to the model (parameterized by $\theta$). Experiments demonstrate that this approach performs on par with MAML while adapting far fewer parameters. An additional benefit is that this approach is less sensitive to the adaptation learning rate and is easier to implement and faster to compute.
Strengths
While not really explained in the paper, this work connects gradient-based to embedding-based meta-learning approaches. Adaptation is via gradient descent, but the adapted parameters are then re-interpreted as an embedding.
The method has the potential to perform on par with MAML while being simpler and faster.
The paper is well-written.
Weaknesses
The field of meta-learning variants is crowded, and this paper struggles to carve out its novelty. 
Rusu et al (LEO) optimize a context vector, which is used to generate model parameters. Reducing the generative model to a point estimate, how is this different from generating the FiLM parameters as a function of context as done in CAML? 
Lee and Choi (MT-nets) propose a general formulation for learning which model parameters to adapt. CAML is simpler in that the model parameters to adapt are chosen beforehand to be inputs. 
Snell et al. / Oreshkin et al. are prototype-based methods infer context via a neural network rather than optimizing for it.
In this context, CAML appears to be yet another point drawn from the convex hull of choices already explored in episodic meta-learning (these choices can be broadly grouped into task encoding and conditional inference). The paper must then rest on its experimental results, which are at present unconvincing.
On the whole, the experimental results seem weak and analysis results largely uninformative. The method is benchmarked on the toy tasks of sinusoid regression and a 2-D point mass, as well as mini-ImageNet few-shot classification. The sinusoid and point mass navigation are toy and compared only to MAML, so it is hard to draw conclusions from those experiments. For mini-ImageNet, while CAML outperforms MAML, it seems that the pertinent comparison is with MT-NET (which CAML does not outperform) and LEO (missing fair comparison?).
Questions regarding experiments
 - CAML is robust to the adaptation learning rate, but isn’t this true of any scheme that separates meta-learned and adapted parameters into disjoint sets? (e.g. also true of Lee and Choi?) 
 - The visualizations of the context parameters are nice, but interpreting much higher dimensional context vectors (which would be necessary for harder tasks) is more difficult, so I’m not sure what to take away from this? It’s very unsurprising that the 2-D context vector encodes x and y position in the point mass experiment, for example. 
 - I am confused by the comparison between adapting input parameters versus subsets of nodes at each layer or entire layers for the sinusoid regression task. Adapting subsets of nodes at each layer roughly corresponds to Lee and Choi, yet the reported numbers are quite different? 
 - In Table 3, which CAML is a fair comparison (in terms of network size and architecture) to MT-NET? 
Editorial Notes
Intro paragraph 3: fine-tuning image classification features for a semantic segmentation task is not a good example of task independent parameters, since fine-tuning end-to-end gives significant improvements.
Related work paragraph 2: Initializing context parameters to zero is not the only difference with Rei et al (2015), and seems a strange thing to highlight?
Tables 1 and 2: state what the task is in the caption