has three maps, one for each color channel (RGB). The CIFAR10 images are relatively small in
comparison to NORB's, and force us to use small kernels. The tested CNNs dier only in the
number of maps per convolutional and max-pooling layer. All have eight hidden layers: layer1, a
convolutional layer with 3 3 kernels and skipping factor of 0; layer2, a max-pooling layer over a
33 region; layer3, a convolutional layer with 3 3 kernels and skipping factors of 0 0; layer4,
a max-pooling over a 2 2 region; layer5, a convolutional layer with 3 3 kernels and a skipping
factors of 0 0; layer6, a max pooling layer over a 2 2 region; layer7, a fully connected layer
with 300 neurons; layer8, a fully connected layer with 100 neurons.
Like for MNIST, the learning rate is initialized by 0.001 and multiplied by 0.993 after every
epoch. Results in Table 3 show that without translation the error rate does not drop below 28%;
adding edge information does not help at all. Translations have a very positive eect, decreasing
We present a fast, fully parameterizable GPU implementation of Convolutional
Neural Network variants. Our feature extractors are neither carefully designed
nor pre-wired, but rather learned in a supervised way. Our deep hierarchical
architectures achieve the best published results on benchmarks for object
classification (NORB, CIFAR10) and handwritten digit recognition (MNIST), with
error rates of 2.53%, 19.51%, 0.35%, respectively. Deep nets trained by simple
back-propagation perform better than more shallow ones. Learning is
surprisingly rapid. NORB is completely trained within five epochs. Test error
rates on MNIST drop to 2.42%, 0.97% and 0.48% after 1, 3 and 17 epochs,
respectively.
Dan C. Cireşan Ueli Meier Jonathan Masci Luca M. Gambardella Jürgen Schmidhuber