5·10−4)anddropoutregularisationfortheﬁrsttwofully-connect edlayers(dropoutratiosetto 0.5).
Thelearningrate wasinitially setto 10−2,andthendecreasedbyafactorof 10whenthevalidation
set accuracy stopped improving. In total, the learning rate was decreased 3 times, and the learning
was stopped after 370K iterations (74 epochs). We conjecture that in spite of the l arger number of
parametersandthegreaterdepthofournetscomparedto(Kri zhevskyetal.,2012),thenetsrequired
lessepochstoconvergedueto(a)implicitregularisationi mposedbygreaterdepthandsmallerconv.
ﬁlter sizes; (b)pre-initialisationofcertainlayers.
The initialisation of the networkweightsis important,sin ce bad initialisation can stall learningdue
to the instability of gradient in deep nets. To circumvent th is problem, we began with training
the conﬁgurationA (Table 1), shallow enoughto be trained wi th randominitialisation. Then,when
In this work we investigate the effect of the convolutional network depth on
its accuracy in the large-scale image recognition setting. Our main
contribution is a thorough evaluation of networks of increasing depth using an
architecture with very small (3x3) convolution filters, which shows that a
significant improvement on the prior-art configurations can be achieved by
pushing the depth to 16-19 weight layers. These findings were the basis of our
ImageNet Challenge 2014 submission, where our team secured the first and the
second places in the localisation and classification tracks respectively. We
also show that our representations generalise well to other datasets, where
they achieve state-of-the-art results. We have made our two best-performing
ConvNet models publicly available to facilitate further research on the use of
deep visual representations in computer vision.
Karen Simonyan Andrew Zisserman