2006.
D. C. Cire san, U. Meier, L. M. Gambardella, and J. Schmidhuber. Deep big simple neural nets
for handwritten digit recogntion. Neural Computation , 22(12):3207{3220, 2010.
A. Coates, H. Lee, and A. Ng. An analysis of single-layer networks in unsupervised feature learning.
InAdvances in Neural Information Processing Systems , 2010.
K. Fukushima. Neocognitron: A self-organizing neural network for a mechanism of pattern recognition unaected by shift in position. Biological Cybernetics , 36(4):193{202, 1980.
K. Fukushima. Neocognitron for handwritten digit recognition. Neurocomputing , 51:161{180,
2003.
P. O. Hoyer and A. Hyv arinen. Independent component analysis applied to feature extraction
from colour and stero images. Network: Computation in Neural Systems , 11(3):191{210, 2000.
A. Krizhevsky. Learning multiple layers of features from tiny images. Master's thesis, Computer
Science Department, University of Toronto, 2009.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haner. Gradient-based learning applied to document
We present a fast, fully parameterizable GPU implementation of Convolutional
Neural Network variants. Our feature extractors are neither carefully designed
nor pre-wired, but rather learned in a supervised way. Our deep hierarchical
architectures achieve the best published results on benchmarks for object
classification (NORB, CIFAR10) and handwritten digit recognition (MNIST), with
error rates of 2.53%, 19.51%, 0.35%, respectively. Deep nets trained by simple
back-propagation perform better than more shallow ones. Learning is
surprisingly rapid. NORB is completely trained within five epochs. Test error
rates on MNIST drop to 2.42%, 0.97% and 0.48% after 1, 3 and 17 epochs,
respectively.
Dan C. Cireşan Ueli Meier Jonathan Masci Luca M. Gambardella Jürgen Schmidhuber