adding edge information does not help at all. Translations have a very positive eect, decreasing
the error rate to almost 20%. Contrast extraction lters are better than the Sobel/Scharr lters but
still worse than no pre-processing layer at all. Despite some CNN-inherent translation invariance,
additional training image translations cause better generalization; additional image processing
proved useless though.
To see if bigger nets are better, we increase the number of maps per layer from 100 to 200, 300
and 400, respectively (last three rows in Tab. 3). Training time increases exponentially, but the
test error decreases, reaching a minimum for nets with 300 maps per layer. Our 19.51% error rate
is better than the previous state of the art for this dataset, 20.40% (Coates et al. , 2010) and 25.50%
Technical Report No. IDSIA-01-11 9
Table 3: Average error rates and standard deviations for N runs of an eight hidden layer CNN on
the CIFAR10 test set (see text for details). The rst ve nets have 100 maps per convolutional
and max-pooling layer, whereas the sixth, seventh and eighth have 200, 300 and 400 maps per
We present a fast, fully parameterizable GPU implementation of Convolutional
Neural Network variants. Our feature extractors are neither carefully designed
nor pre-wired, but rather learned in a supervised way. Our deep hierarchical
architectures achieve the best published results on benchmarks for object
classification (NORB, CIFAR10) and handwritten digit recognition (MNIST), with
error rates of 2.53%, 19.51%, 0.35%, respectively. Deep nets trained by simple
back-propagation perform better than more shallow ones. Learning is
surprisingly rapid. NORB is completely trained within five epochs. Test error
rates on MNIST drop to 2.42%, 0.97% and 0.48% after 1, 3 and 17 epochs,
respectively.
Dan C. Cireşan Ueli Meier Jonathan Masci Luca M. Gambardella Jürgen Schmidhuber