optimizeroptimizeeparameter updates
error signalFigure 1: The optimizer (left) is provided with
performance of the optimizee (right) and proposes
updates to increase the optimizee’s performance.
[photos: Bobolas, 2009, Maley, 2011]This industry of optimizer design allows different communities to create optimization methods which exploit structure in their problems
of interest at the expense of potentially poor
performance on problems outside of that scope.
Moreover the No Free Lunch Theorems for Optimization [Wolpert and Macready, 1997] show
that in the setting of combinatorial optimization,
no algorithm is able to do better than a random
strategy in expectation. This suggests that specialization to a subclass of problems is in fact
theonly way that improved performance can be
achieved in general.
In this work we take a different tack and instead
propose to replace hand-designed update rules
with a learned update rule, which we call the optimizerg, speciﬁed by its own set of parameters
. This results in updates to the optimizee fof
the form
t+1=t+gt(rf(t);): (1)
A high level view of this process is shown in Figure 1. In what follows we will explicitly model
The move from hand-designed features to learned features in machine learning
has been wildly successful. In spite of this, optimization algorithms are still
designed by hand. In this paper we show how the design of an optimization
algorithm can be cast as a learning problem, allowing the algorithm to learn to
exploit structure in the problems of interest in an automatic way. Our learned
algorithms, implemented by LSTMs, outperform generic, hand-designed competitors
on the tasks for which they are trained, and also generalize well to new tasks
with similar structure. We demonstrate this on a number of tasks, including
simple convex problems, training neural networks, and styling images with
neural art.
Marcin Andrychowicz Misha Denil Sergio Gomez Matthew W. Hoffman David Pfau Tom Schaul Brendan Shillingford Nando de Freitas