Learning an efficient update rule from data that promotes rapid learning of new tasks from the same distribution remains an open problem in meta-learning. Typically, previous works have approached this issue either by attempting to train a neural network that directly produces updates or by attempting to learn better initialisations or scaling factors for a gradient-based update rule. Both these approaches pose challenges. On one hand, directly producing an update forgoes a useful inductive bias and can easily lead to non-converging behaviour. On the other hand, approaches that try to control a gradient-based update rule typically resort to computing gradients through the learning process to obtain their metagradients, leading to methods that can not scale beyond few-shot task adaptation. In this work we propose Warped Gradient Descent (WarpGrad), a method that intersects these approaches to mitigate their limitations. WarpGrad meta-learns an efficiently parameterised preconditioning matrix that facilitates gradient descent across the task distribution. Preconditioning arises by interleaving non-linear layers, referred to as warp-layers, between the layers of a task-learner. Warp-layers are meta-learned without backpropagating through the task training process in a manner similar to methods that learn to directly produce updates. WarpGrad is computationally efficient, easy to implement, and can scale to arbitrarily large meta-learning problems. We provide a geometrical interpretation of the approach and evaluate its effectiveness in a variety of settings, including few-shot, standard supervised, continual and reinforcement learning. Learning (how)  to learn implies inferring a learning strategy from some set of past experiences via a meta-learner that a task-learner can leverage when learning a new task. One approach is to directly parameterise an update rule via the memory of a recurrent neural network (Andrychowicz et al.,  2016; Ravi & Larochelle, 2016; Li & Malik, 2016; Chen et al., 2017). Such memory-based methods can, in principle, represent any learning rule by virtue of being universal function approximators (Cybenko, 1989; Hornik, 1991;Schäfer & Zimmermann, 2007). They can also scale to long learning processes by using truncated backpropagation through time, but they lack an inductive bias as to what constitutes a reasonable learning rule. This renders them hard to train and brittle to generalisation as their parameter updates have no guarantees of convergence.
Summary:
The current paper deals with meta-learning and essentially proposes a generalization of MAML (a popular gradient-based meta-learning algorithm) that mostly builds upon two main recent advances in meta-learning: 1) an architectural one (see e.g. T-Nets), which consists in optimizing the parameters of additional layers during the meta-learning outer loop (as opposed to only optimizing the initial conditions of the original parameters like in MAML), and 2) a theoretical one (see e.g. Meta-SGD, Meta-curvature), which is based on the geometrical observation that one set of parameters can precondition a second set of parameters that are consequently being optimized in a "warped" geometry, possibly speeding up learning.
The authors provide a great and thorough overview of the literature, in particular for gradient-based meta-learning methods, which helps putting all this in perspective.
The way they obtain the mentioned "warped" geometry in practice is by adding additional so-called warp-layers to an architecture that is being trained with meta-learning. Such warp-layer are generic deep learning modules (such as convolutions followed by BatchNorm, or LSTM layers), which are being trained in the outer-loop of the meta-learning optimization. In this sense, WarpGrad extend T-Nets, which only allowed for linear layers.
The second main innovation of WarpGrad is the proposal of a new meta-learning objective, which incorporates a meta-learning internal loop of only one step of (preconditioned) SGD, meaning that, as the authors notes, "in contrast to MAML-based approaches (Eq. 1), [...] avoids backpropagation through learning processes".
The authors test their algorithm on several meta-learning benchmarks, including few- and multi-shot learning tasks demonstrating very competitive performance when their algorithm is combined with MAML or Leap. They then deploy WarpGrad on a maze navigation reinforcemente learning task to demonstrate training of recurrent architectures, and on a continual learning toy dataset to show that their objective can be adapted to mitigate catastrophic forgetting. 
Decision:
This is a good paper which proposes an interesting generalization of previous gradient-based meta-learning methods like MAML and T-Net, with an impressive number of experiments. However, some of the statements regarding the advantages of WarpGrad over previous algorithms seem a little bit misleading, in particular in situations where WarpGrad needs to be combined with these same algorithms. For instance (and I might have completely misunderstood things here), it seems that when the WarpGrad objective is being combined with MAML (which requires backpropagation through multiple-step gradient descent trajectories), then also the resulting combined objective will necessarily need to backprop through the same multi-step trajectory, defeating the stated advantage of the WarpGrad algorithm (i.e. that its objective avoids backpropagating through the learning processes).
In general, even if one only considers the WarpGrad objective eq. (10), that comprises a meta-learning inner loop which consists of one step of (preconditioned) gradient descent. However, it seems like an arbitrary (and limiting) choice of the authors to only perform one step, as opposed to multiple ones. As a matter of fact, even very sophisticated second order gradient descent methods like natural gradient descent typically require more than one step to reach a local minimum. That is to say, that the main advantage showcased by the authors (the fact that the WarpGrad objective avoids backprop through a whole learning trajectory) seems like a limitation, rather than the result of a principled derivation.
It would be beneficial if the authors could clarify this points. In particular, whether combining WarpGrad with MAML does not indeed negate the stated advantages of WarpGrad over MAML, and whether there is a principled way of demonstrating that executing only one step in the inner loop of the WarpGrad objective is completely general (i.e., additional steps do not help the inner loop).
Minor:
- The authors use the wrong citation key when referring to the T-net paper: it should be Lee et al 2018, instead of Lee et al. 2017
- I believe that when the authors mention Fast and slow weights, they are being described in the opposite way: slow weights should be in charge of meta-learning information, while fast ones are in charge of task-specific information.
- Line 3 and 4 of Algorithm 1 and 2: shouldn't it say "mini-batch of tasks" (plural), instead of "mini-batch of task", since several tasks are being sampled? Otherwise, it might be erroneously interpreted as "mini-batch of (samples belonging to) task T".
- The comment that "learning to precondition gradients can be seen as a Markov Process of order 1" is never clearly elucidated or developed. It would help to develop this.