y(Sy+ 1) jy(Sy+ 1) +Ky1:
We can now compute the inequalities for ( x;y):
iKx+ 1
Sx+ 1xi
Sx+ 1;
jKy+ 1
Sy+ 1yj
Sy+ 1:
Because (x;y) has to be inside the map, the nal inequalities are:
maxiKx+ 1
Sx+ 1
;0
xmini
Sx+ 1
;Mx1
;
maxjKy+ 1
Sy+ 1
;0
yminj
Sy+ 1
;My1
:
The above inequalities state that the delta of neuron ( i;j) fromLn1is computed from deltas of
neurons in a rectangular area in maps of Ln(Fig. 2). After summing up the deltas, each thread
multiplies the result by the derivative of the activation function.
3.4 Adjusting weights
FP and BP have a grid on the list of maps, but the AW thread grid is on the list of kernels (lters)
We present a fast, fully parameterizable GPU implementation of Convolutional
Neural Network variants. Our feature extractors are neither carefully designed
nor pre-wired, but rather learned in a supervised way. Our deep hierarchical
architectures achieve the best published results on benchmarks for object
classification (NORB, CIFAR10) and handwritten digit recognition (MNIST), with
error rates of 2.53%, 19.51%, 0.35%, respectively. Deep nets trained by simple
back-propagation perform better than more shallow ones. Learning is
surprisingly rapid. NORB is completely trained within five epochs. Test error
rates on MNIST drop to 2.42%, 0.97% and 0.48% after 1, 3 and 17 epochs,
respectively.
Dan C. Cireşan Ueli Meier Jonathan Masci Luca M. Gambardella Jürgen Schmidhuber