aslearning to learn without gradient descent by gradient descent . The work of Runarsson and
Jonsson [2000] builds upon this work by replacing the simple rule with a neural network.
Cotter and Conwell [1990], and later Younger et al. [1999], also show ﬁxed-weight recurrent neural
networks can exhibit dynamic behavior without need to modify their network weights. Similarly this
has been shown in a ﬁltering context [e.g. Feldkamp and Puskorius, 1998], which is directly related
to simple multi-timescale optimizers [Sutton, 1992, Schraudolph, 1999].
Finally, the work of Younger et al. [2001] and Hochreiter et al. [2001] connects these different threads
of research by allowing for the output of backpropagation from one network to feed into an additional
learning network, with both networks trained jointly. Our approach to meta-learning builds on this
work by modifying the network architecture of the optimizer in order to scale this approach to larger
neural-network optimization problems.
2 Learning to learn with recurrent neural networks
In this work we consider directly parameterizing the optimizer. As a result, in a slight abuse of notation
we will write the ﬁnal optimizee parameters (f;)as a function of the optimizer parameters and
The move from hand-designed features to learned features in machine learning
has been wildly successful. In spite of this, optimization algorithms are still
designed by hand. In this paper we show how the design of an optimization
algorithm can be cast as a learning problem, allowing the algorithm to learn to
exploit structure in the problems of interest in an automatic way. Our learned
algorithms, implemented by LSTMs, outperform generic, hand-designed competitors
on the tasks for which they are trained, and also generalize well to new tasks
with similar structure. We demonstrate this on a number of tasks, including
simple convex problems, training neural networks, and styling images with
neural art.
Marcin Andrychowicz Misha Denil Sergio Gomez Matthew W. Hoffman David Pfau Tom Schaul Brendan Shillingford Nando de Freitas