the best network achieved a recognition test error rate of 0.35%, on NORB 2.53% and on CIFAR10
19.51%. Our results are raising the bars for all three benchmarks. Currently the particular CNN
types discussed in this paper seem to be the best adaptive image recognizers, provided there is a
labeled dataset of sucient size. No unsupervised pretraining is required. Good results require
big and deep but sparsely connected CNNs, computationally prohibitive on CPUs, but feasible on
current GPUs, where our implementation is 10 to 60 times faster than a compiler-optimized CPU
version.
Technical Report No. IDSIA-01-11 10
Acknowledgment
This work was partially funded by the Swiss Commission for Technology and Innovation (CTI),
Project n. 9688.1 IFF: Intelligent Fill in Form.
References
S. Behnke. Hierarchical Neural Networks for Image Interpretation , volume 2766 of Lecture Notes
in Computer Science . Springer, 2003.
K. Chellapilla, S. Puri, and P. Simard. High performance convolutional neural networks for
document processing. In International Workshop on Frontiers in Handwriting Recognition ,
2006.
D. C. Cire san, U. Meier, L. M. Gambardella, and J. Schmidhuber. Deep big simple neural nets
We present a fast, fully parameterizable GPU implementation of Convolutional
Neural Network variants. Our feature extractors are neither carefully designed
nor pre-wired, but rather learned in a supervised way. Our deep hierarchical
architectures achieve the best published results on benchmarks for object
classification (NORB, CIFAR10) and handwritten digit recognition (MNIST), with
error rates of 2.53%, 19.51%, 0.35%, respectively. Deep nets trained by simple
back-propagation perform better than more shallow ones. Learning is
surprisingly rapid. NORB is completely trained within five epochs. Test error
rates on MNIST drop to 2.42%, 0.97% and 0.48% after 1, 3 and 17 epochs,
respectively.
Dan C. Cireşan Ueli Meier Jonathan Masci Luca M. Gambardella Jürgen Schmidhuber