Meta-learning has recently achieved success in many optimization problems. In general, a meta learner g(.) could be learned for a base model f (.) on a variety of tasks, such that it can be more efficient on a new task. In this paper, we make some key modifications to enhance the performance of meta-learning models. (1) we leverage different meta-strategies for different modules to optimize them separately: we use conservative "slow learners" on low-level basic feature representation layers and "fast learners" on high-level task-specific layers; (2) Furthermore, we provide theoretical analysis on why the proposed approach works, based on a case study on a two-layer MLP. We evaluate our model on synthetic MLP regression, as well as low-shot learning tasks on Omniglot and ImageNet benchmarks. We demonstrate that our approach is able to achieve state-of-the-art performance.
[Summary]
The paper presents a novel learning framework for meta-learning that is motivated by neural learning process of human over long periods. Specifically, the process of meta-learning is divided into a slow and a fast learning modules, where the slowly-learnt component accounts for low-level representation that is progressively optimized over all data seen so far to achieve generalization power, and the fastly-learnt component is supposed to pick up the target in a new task for quick adaptation. It is proposed that meta-learning should focus on capturing the meta-information for the fast learning module, and leave the slow module being updated steadily without task-specific adaptation. Theoretical analysis is presented on a linear MLP examples to shed some light on the properties of the proposed algorithm. Results on both synthetic dataset and benchmarks justify the theoretical observation and advantages.               
Pros
Novel treatment and formulation of meta-learning from the perspective of fast and slow  learning process
Cons
Some interesting cases not tested
Presentation could be improved 
[Originality]
The paper approaches the recently popular meta-learning from a novel perspective by decomposing the learning process into slow and fast ones. 
[Quality]
Overall,  the paper is well motivated and implemented with both theoretical study and empirical justification. There are a few questions / areas for further improvements, though:
- It seems that to initialize the slow module, another set of data is needed to pretrain it before the actual meta-learning takes place to learn to optimize the fast learner (as opposed to other meta-learning methods where all parameters in a base model were meta-learnt over the meta training set). How does this affect the performance? E.g., what if the slow module is only updated over the meta-training set (still without reinitialization across different batches) without pre-training?
- In the current formulation, the base model is decomposed into two distinct (slow and fast) modules. What is the rule to decide which layers should belong to slow or fast modules? How does different choice affect the performance? Can we decompose the base model into finer granularities for different learning behaviors? E.g., a third module module in-between the fast and slow ones that follows medium learning pace.          
- The theoretical study can be better organized. The proofs can be left in appendix to make room for more discussion on conclusions, non-linear and / or non-Gaussian cases.     
- The write-up can be improved too at some places: proper reference at line 4 of section 1 is missing; \phi in (1) is not well defined, as well as “SOA” in section 2;
[Clarity]
The paper is generally clearly written, with a few places to improve (see comments above).
[Significance]
The paper brings in an interesting perspective to meta-learning. It can also inspire more follow-up work to better understand the problem.