Backpropagation Through Time (BPTT) inefﬁcient. We solve this problem by relaxing the objective
such thatwt>0at intermediate points along the trajectory. This changes the objective function, but
allows us to train the optimizer on partial trajectories. For simplicity, in all our experiments we use
wt= 1for everyt.
3
OptimizeeOptimizert-2t-1t
mmm+++ft-1ftft-2∇t-2∇t-1∇tht-2ht-1htht+1gt-1gtθt-2θt-1θtθt+1gt-2Figure 2: Computational graph used for computing the gradient of the optimizer.
2.1 Coordinatewise LSTM optimizer
One challenge in applying RNNs in our setting is that we want to be able to optimize at least tens of
thousands of parameters. Optimizing at this scale with a fully connected RNN is not feasible as it
would require a huge hidden state and an enormous number of parameters. To avoid this difﬁculty we
will use an optimizer mwhich operates coordinatewise on the parameters of the objective function,
similar to other common update rules like RMSprop and ADAM. This coordinatewise network
The move from hand-designed features to learned features in machine learning
has been wildly successful. In spite of this, optimization algorithms are still
designed by hand. In this paper we show how the design of an optimization
algorithm can be cast as a learning problem, allowing the algorithm to learn to
exploit structure in the problems of interest in an automatic way. Our learned
algorithms, implemented by LSTMs, outperform generic, hand-designed competitors
on the tasks for which they are trained, and also generalize well to new tasks
with similar structure. We demonstrate this on a number of tasks, including
simple convex problems, training neural networks, and styling images with
neural art.
Marcin Andrychowicz Misha Denil Sergio Gomez Matthew W. Hoffman David Pfau Tom Schaul Brendan Shillingford Nando de Freitas