(Fukushima, 2003) helps to improve the recognition rate for NORB.
2.2 Convolutional layer
A convolutional layer is parametrized by the size and the number of the maps, kernel sizes, skipping
factors, and the connection table. Each layer has Mmaps of equal size ( Mx,My). A kernel (blue
rectangle in Fig 1) of size ( Kx,Ky) is shifted over the valid region of the input image (i.e. the
kernel has to be completely inside the image). The skipping factors SxandSydene how many
pixels the lter/kernel skips in x- and y-direction between subsequent convolutions. The size of
the output map is then dened as:
Mn
x=Mn1
xKn
x
Snx+ 1+ 1;Mn
y=Mn1
yKn
y
Sny+ 1+ 1 (1)
where index nindicates the layer. Each map in layer Lnis connected to at most Mn1maps in
layerLn1. Neurons of a given map share their weights but have dierent receptive elds.
2.3 Max-pooling layer
The biggest architectural dierence between our implementation and the CNN of LeCun et al.
We present a fast, fully parameterizable GPU implementation of Convolutional
Neural Network variants. Our feature extractors are neither carefully designed
nor pre-wired, but rather learned in a supervised way. Our deep hierarchical
architectures achieve the best published results on benchmarks for object
classification (NORB, CIFAR10) and handwritten digit recognition (MNIST), with
error rates of 2.53%, 19.51%, 0.35%, respectively. Deep nets trained by simple
back-propagation perform better than more shallow ones. Learning is
surprisingly rapid. NORB is completely trained within five epochs. Test error
rates on MNIST drop to 2.42%, 0.97% and 0.48% after 1, 3 and 17 epochs,
respectively.
Dan C. Cireşan Ueli Meier Jonathan Masci Luca M. Gambardella Jürgen Schmidhuber