Broyden, Fletcher, Goldfarb, and Shanno. We call this architecture an NTM-BFGS optimizer , because
its use of external memory is similar to the Neural Turing Machine [Graves et al., 2014]. The pivotal
differences between our construction and the NTM are (1) our memory allows only low-rank updates;
(2) the controller (including read/write heads) operates coordinatewise.
In BFGS an explicit estimate of the full (inverse) Hessian is built up from the sequence of observed
gradients. We can write a skeletonized version of the BFGS algorithm, using Mtto represent the
inverse Hessian approximation at iteration t, as follows
gt= read(Mt;t)
t+1=t+gt
Mt+1= write(Mt;t;gt):
Here we have packed up all of the details of the BFGS algorithm into the suggestively named read
andwrite operations, which operate on the inverse Hessian approximation Mt. In BFGS these
operations have speciﬁc forms, for example read(Mt;t) =Mtrf(t)is a speciﬁc matrix-vector
multiplication and the BFGS write operation corresponds to a particular low-rank update of Mt.
The move from hand-designed features to learned features in machine learning
has been wildly successful. In spite of this, optimization algorithms are still
designed by hand. In this paper we show how the design of an optimization
algorithm can be cast as a learning problem, allowing the algorithm to learn to
exploit structure in the problems of interest in an automatic way. Our learned
algorithms, implemented by LSTMs, outperform generic, hand-designed competitors
on the tasks for which they are trained, and also generalize well to new tasks
with similar structure. We demonstrate this on a number of tasks, including
simple convex problems, training neural networks, and styling images with
neural art.
Marcin Andrychowicz Misha Denil Sergio Gomez Matthew W. Hoffman David Pfau Tom Schaul Brendan Shillingford Nando de Freitas