we will write the ﬁnal optimizee parameters (f;)as a function of the optimizer parameters and
the function in question. We can then ask the question: What does it mean for an optimizer to be
good? Given a distribution of functions fwe will write the expected loss as
L() =Efh
f
(f;)i
: (2)
As noted earlier, we will take the update steps gtto be the output of a recurrent neural network m,
parameterized by , whose state we will denote explicitly with ht. Next, while the objective function
in(2)depends only on the ﬁnal parameter value, for training the optimizer it will be convenient to
have an objective that depends on the entire trajectory of optimization, for some horizon T,
L() =Ef"TX
t=1wtf(t)#
where t+1=t+gt;
gt
ht+1
=m(rt;ht;):(3)
Herewt2R0are arbitrary weights associated with each time-step and we will also use the notation
rt=rf(t). This formulation is equivalent to (2)whenwt=1[t=T], but later we will describe
The move from hand-designed features to learned features in machine learning
has been wildly successful. In spite of this, optimization algorithms are still
designed by hand. In this paper we show how the design of an optimization
algorithm can be cast as a learning problem, allowing the algorithm to learn to
exploit structure in the problems of interest in an automatic way. Our learned
algorithms, implemented by LSTMs, outperform generic, hand-designed competitors
on the tasks for which they are trained, and also generalize well to new tasks
with similar structure. We demonstrate this on a number of tasks, including
simple convex problems, training neural networks, and styling images with
neural art.
Marcin Andrychowicz Misha Denil Sergio Gomez Matthew W. Hoffman David Pfau Tom Schaul Brendan Shillingford Nando de Freitas