multiplication and the BFGS write operation corresponds to a particular low-rank update of Mt.
In this work we preserve the structure of the BFGS updates, but discard their particular form. More
speciﬁcally the read operation remains a matrix-vector multiplication but the form of the vector
used is learned. Similarly, the write operation remains a low-rank update, but the vectors involved
are also learned. Conveniently, this structure of interaction with a large dynamically updated state
corresponds in a fairly direct way to the architecture of a Neural Turing Machine (NTM), where Mt
corresponds to the NTM memory [Graves et al., 2014].
Our NTM-BFGS optimizer uses an LSTM+GAC as a controller; however, instead of producing the
update directly we attach one or more read and write heads to the controller. Each read head produces
a read vector rtwhich is combined with the memory to produce a read result itwhich is fed back
into the controller at the following time step. Each write head produces two outputs, a left write
vectoratand a right write vector bt. The two write vectors are used to update the memory state by
accumulating their outer product. The read and write operation for a single head is diagrammed in
Figure 14 and the way read and write heads are attached to the controller is depicted in Figure 15.
The move from hand-designed features to learned features in machine learning
has been wildly successful. In spite of this, optimization algorithms are still
designed by hand. In this paper we show how the design of an optimization
algorithm can be cast as a learning problem, allowing the algorithm to learn to
exploit structure in the problems of interest in an automatic way. Our learned
algorithms, implemented by LSTMs, outperform generic, hand-designed competitors
on the tasks for which they are trained, and also generalize well to new tasks
with similar structure. We demonstrate this on a number of tasks, including
simple convex problems, training neural networks, and styling images with
neural art.
Marcin Andrychowicz Misha Denil Sergio Gomez Matthew W. Hoffman David Pfau Tom Schaul Brendan Shillingford Nando de Freitas