and max-pooling layer, whereas the sixth, seventh and eighth have 200, 300 and 400 maps per
hidden layer, respectively. IP - image processing layer: edge - 3 3 Sobel and Scharr lters; hat 1313 positive and negative contrast extraction lters.
trans. [%] maps IP TfbV [%] runs time/epoch [s]
0 100 no 28.87 0.37 11 93
0 100 edge 29.11 0.36 15 104
5 100 no 20.26 0.21 11 111
5 100 edge 21.87 0.57 5 120
5 100 hat 21.44 0.44 4 136
5 200 no 19.90 0.16 5 248
5 300 no 19.51 0.18 5 532
5 400 no 19.54 0.16 5 875
(Yu and Zhang, 2010). Unlike Coates et al. (2010), however, we use the original images without
any particular input normalization. Note that the error rate standard deviations are smaller than
those obtained on NORB, that is, dierent initializations yield consistent results.
4.4 Speedup factor of GPU code
The GPU code scales well with network size. For small nets the speedup is small (but still over 10)
We present a fast, fully parameterizable GPU implementation of Convolutional
Neural Network variants. Our feature extractors are neither carefully designed
nor pre-wired, but rather learned in a supervised way. Our deep hierarchical
architectures achieve the best published results on benchmarks for object
classification (NORB, CIFAR10) and handwritten digit recognition (MNIST), with
error rates of 2.53%, 19.51%, 0.35%, respectively. Deep nets trained by simple
back-propagation perform better than more shallow ones. Learning is
surprisingly rapid. NORB is completely trained within five epochs. Test error
rates on MNIST drop to 2.42%, 0.97% and 0.48% after 1, 3 and 17 epochs,
respectively.
Dan C. Cireşan Ueli Meier Jonathan Masci Luca M. Gambardella Jürgen Schmidhuber