January 2011
Abstract
We present a fast, fully parameterizable GPU implementation of Convolutional Neural
Network variants. Our feature extractors are neither carefully designed nor pre-wired, but
rather learned in a supervised way. Our deep hierarchical architectures achieve the best
published results on benchmarks for object classication (NORB, CIFAR10) and handwritten
digit recognition (MNIST), with error rates of 2.53%, 19.51%, 0.35%, respectively. Deep
nets trained by simple back-propagation perform better than more shallow ones. Learning
is surprisingly rapid. NORB is completely trained within ve epochs. Test error rates on
MNIST drop to 2.42%, 0.97% and 0.48% after 1, 3 and 17 epochs, respectively.
1 Introduction
The human visual system eciently recognizes and localizes objects within cluttered scenes. For
articial systems, however, this is still dicult, due to viewpoint-dependent object variability,
and the high in-class variability of many object types. Deep hierarchical neural models roughly
mimick the nature of mammalian visual cortex, and by community consensus are among the most
promising architectures for such tasks. The most successful hierarchical object recognition systems
all extract localized features from input images, convolving image patches with lters. Filter
We present a fast, fully parameterizable GPU implementation of Convolutional
Neural Network variants. Our feature extractors are neither carefully designed
nor pre-wired, but rather learned in a supervised way. Our deep hierarchical
architectures achieve the best published results on benchmarks for object
classification (NORB, CIFAR10) and handwritten digit recognition (MNIST), with
error rates of 2.53%, 19.51%, 0.35%, respectively. Deep nets trained by simple
back-propagation perform better than more shallow ones. Learning is
surprisingly rapid. NORB is completely trained within five epochs. Test error
rates on MNIST drop to 2.42%, 0.97% and 0.48% after 1, 3 and 17 epochs,
respectively.
Dan C. Cireşan Ueli Meier Jonathan Masci Luca M. Gambardella Jürgen Schmidhuber