to discovering good forms of biologically inspired visual representation. PLoS computational
biology , 5(11):e1000579, November 2009.
M. Riesenhuber and T. Poggio. Hierarchical models of object recognition in cortex. Nat. Neurosci. ,
2(11):1019{1025, 1999.
Technical Report No. IDSIA-01-11 11
D. Scherer, A. M uller, and S. Behnke. Evaluation of pooling operations in convolutional architectures for object recognition. In International Conference on Articial Neural Networks , 2010.
J. Schmidhuber, M. Eldracher, and B. Foltin. Semilinear predictability minimization produces
well-known feature detectors. Neural Computation , 8(4):773{786, 1996.
J. Schmidhuber. Learning factorial codes by predictability minimization. Neural Computation ,
4(6):863{879, 1992.
T. Serre, L. Wolf, and T. Poggio. Object recognition with features inspired by visual cortex. In
Proc. of Computer Vision and Pattern Recognition Conference , 2007.
P. Simard, D. Steinkraus, and J. Platt. Best practices for convolutional neural networks applied
to visual document analysis. In Seventh International Conference on Document Analysis and
We present a fast, fully parameterizable GPU implementation of Convolutional
Neural Network variants. Our feature extractors are neither carefully designed
nor pre-wired, but rather learned in a supervised way. Our deep hierarchical
architectures achieve the best published results on benchmarks for object
classification (NORB, CIFAR10) and handwritten digit recognition (MNIST), with
error rates of 2.53%, 19.51%, 0.35%, respectively. Deep nets trained by simple
back-propagation perform better than more shallow ones. Learning is
surprisingly rapid. NORB is completely trained within five epochs. Test error
rates on MNIST drop to 2.42%, 0.97% and 0.48% after 1, 3 and 17 epochs,
respectively.
Dan C. Cireşan Ueli Meier Jonathan Masci Luca M. Gambardella Jürgen Schmidhuber