image ,s, and is given by
f() =Lcontent (c;) +Lstyle(s;) +
Lreg()
The minimizer of fis the styled image . The ﬁrst two terms try to match the content and style of
the styled image to that of their ﬁrst argument, and the third term is a regularizer that encourages
smoothness in the styled image. Details can be found in [Gatys et al., 2015].
7
20 40 60 80 100 120
StepLossNeural art, training resolution
20 40 60 80 100 120
StepDouble resolution
ADAM
RMSprop
SGD
NAG
LSTMFigure 8: Optimization curves for Neural Art. Content images come from the test set, which was not
used during the LSTM optimizer training. Note: the y-axis is in log scale and we zoom in on the
interesting portion of this plot. Left: Applying the training style at the training resolution. Right:
Applying the test style at double the training resolution.
Figure 9: Examples of images styled using the LSTM optimizer. Each triple consists of the content
image (left), style (right) and image generated by the LSTM optimizer (center). Left: The result of
The move from hand-designed features to learned features in machine learning
has been wildly successful. In spite of this, optimization algorithms are still
designed by hand. In this paper we show how the design of an optimization
algorithm can be cast as a learning problem, allowing the algorithm to learn to
exploit structure in the problems of interest in an automatic way. Our learned
algorithms, implemented by LSTMs, outperform generic, hand-designed competitors
on the tasks for which they are trained, and also generalize well to new tasks
with similar structure. We demonstrate this on a number of tasks, including
simple convex problems, training neural networks, and styling images with
neural art.
Marcin Andrychowicz Misha Denil Sergio Gomez Matthew W. Hoffman David Pfau Tom Schaul Brendan Shillingford Nando de Freitas