promising architectures for such tasks. The most successful hierarchical object recognition systems
all extract localized features from input images, convolving image patches with lters. Filter
responses are then repeatedly sub-sampled and re-ltered, resulting in a deep feed-forward network
architecture whose output feature vectors are eventually classied. One of the rst hierarchical
neural systems was the Neocognitron (Fukushima, 1980) which inspired many of the more recent
variants.
Unsupervised learning methods applied to patches of natural images tend to produce localized
lters that resemble o-center-on-surround lters, orientation-sensitive bar detectors, Gabor lters
(Schmidhuber et al. , 1996; Olshausen and Field, 1997; Hoyer and Hyv arinen, 2000). These ndings
in conjunction with experimental studies of the visual cortex justify the use of such lters in the
so-called standard model for object recognition (Riesenhuber and Poggio, 1999; Serre et al. , 2007;
Mutch and Lowe, 2008), whose lters are xed, in contrast to those of Convolutional Neural
We present a fast, fully parameterizable GPU implementation of Convolutional
Neural Network variants. Our feature extractors are neither carefully designed
nor pre-wired, but rather learned in a supervised way. Our deep hierarchical
architectures achieve the best published results on benchmarks for object
classification (NORB, CIFAR10) and handwritten digit recognition (MNIST), with
error rates of 2.53%, 19.51%, 0.35%, respectively. Deep nets trained by simple
back-propagation perform better than more shallow ones. Learning is
surprisingly rapid. NORB is completely trained within five epochs. Test error
rates on MNIST drop to 2.42%, 0.97% and 0.48% after 1, 3 and 17 epochs,
respectively.
Dan C. Cireşan Ueli Meier Jonathan Masci Luca M. Gambardella Jürgen Schmidhuber