Experience with other image datasets tells us that NORB is unusual. The training set has only
ve instances per class. The resulting poor training set variability makes the nets learn quickly
but generalize badly. NORB is the only dataset that prots from a xed pre-processing layer in
a substantial way. For MNIST and CIFAR10 such pre-processing has little or no eect. It is also
worth noting that NORB's standard error rate deviation is bigger than CIFAR10's (see Tables 2
and 3). Identical nets with dierent initializations do not produce very consistent results. The
best net had an error rate of 1.72%, the worst 3.69%.
4.3 Experiments on CIFAR 10
CIFAR10 is a collection of natural color images of 32x32 pixels. It contains 10 classes, each of them
with 5000 samples in the training set and 1000 in the test set. The images greatly vary inside each
class. They are not necessarily centered, may contain only parts of the object, and have varying
backgrounds. All of this makes CIFAR10 the hardest problem addressed in this paper. The CNN
has three maps, one for each color channel (RGB). The CIFAR10 images are relatively small in
We present a fast, fully parameterizable GPU implementation of Convolutional
Neural Network variants. Our feature extractors are neither carefully designed
nor pre-wired, but rather learned in a supervised way. Our deep hierarchical
architectures achieve the best published results on benchmarks for object
classification (NORB, CIFAR10) and handwritten digit recognition (MNIST), with
error rates of 2.53%, 19.51%, 0.35%, respectively. Deep nets trained by simple
back-propagation perform better than more shallow ones. Learning is
surprisingly rapid. NORB is completely trained within five epochs. Test error
rates on MNIST drop to 2.42%, 0.97% and 0.48% after 1, 3 and 17 epochs,
respectively.
Dan C. Cireşan Ueli Meier Jonathan Masci Luca M. Gambardella Jürgen Schmidhuber