Figure 14 and the way read and write heads are attached to the controller is depicted in Figure 15.
In can be shown that NTM-BFGS with one read head and 3 write heads can simulate inverse Hessian
BFGS assuming that the controller can compute arbitrary (coordinatewise) functions and have access
to 2 GACs.
NTM-L-BFGS optimizer
In cases where memory is constrained we can follow the example of L-BFGS and maintain a low
rank approximation of the full memory ( vis. inverse Hessian). The simplest way to do this is to store a
sliding history of the left and right write vectors, allowing us to form the matrix vector multiplication
required by the read operation efﬁciently.
16
Controller         ReadWriteMtrtht+1Mt+1rt+1itatbt∇thtLSTM1LSTM2…gt
LSTMk∇kikakbkrkhkhkgkFigure 15: Left: Interaction between the controller and the external memory in NTM-BFGS. The
controller is composed of replicated coordinatewise LSTMs (possibly with GACs), but the read and
write operations are global across all coordinates. Right: A single LSTM for the kth coordinate in
The move from hand-designed features to learned features in machine learning
has been wildly successful. In spite of this, optimization algorithms are still
designed by hand. In this paper we show how the design of an optimization
algorithm can be cast as a learning problem, allowing the algorithm to learn to
exploit structure in the problems of interest in an automatic way. Our learned
algorithms, implemented by LSTMs, outperform generic, hand-designed competitors
on the tasks for which they are trained, and also generalize well to new tasks
with similar structure. We demonstrate this on a number of tasks, including
simple convex problems, training neural networks, and styling images with
neural art.
Marcin Andrychowicz Misha Denil Sergio Gomez Matthew W. Hoffman David Pfau Tom Schaul Brendan Shillingford Nando de Freitas