full-size (uncropped) images at multiple scales (as descri bed above). Multi-GPU training exploits
data parallelism, and is carried out by splitting each batch of training images into several GPU
batches, processed in parallel on each GPU. After the GPU bat ch gradientsare computed, they are
averaged to obtain the gradient of the full batch. Gradient c omputation is synchronous across the
GPUs, sothe resultisexactlythesame aswhentrainingona si ngleGPU.
While more sophisticated methods of speeding up ConvNet tra ining have been recently proposed (Krizhevsky, 2014), which employmodeland data paral lelism for differentlayersof the net,
wehavefoundthatourconceptuallymuchsimplerschemealre adyprovidesaspeedupof 3.75times
on an off-the-shelf4-GPU system, as comparedto using a sing le GPU. On a system equippedwith
fourNVIDIATitanBlackGPUs,trainingasinglenettook2–3w eeksdependingonthearchitecture.
4 CLASSIFICATION EXPERIMENTS
Dataset. In this section, we present the image classiﬁcation results achieved by the described
In this work we investigate the effect of the convolutional network depth on
its accuracy in the large-scale image recognition setting. Our main
contribution is a thorough evaluation of networks of increasing depth using an
architecture with very small (3x3) convolution filters, which shows that a
significant improvement on the prior-art configurations can be achieved by
pushing the depth to 16-19 weight layers. These findings were the basis of our
ImageNet Challenge 2014 submission, where our team secured the first and the
second places in the localisation and classification tracks respectively. We
also show that our representations generalise well to other datasets, where
they achieve state-of-the-art results. We have made our two best-performing
ConvNet models publicly available to facilitate further research on the use of
deep visual representations in computer vision.
Karen Simonyan Andrew Zisserman