Technical Report No. IDSIA-01-11 5
The (uni- or bi-dimensional) thread grid assigns a (bi- or uni-dimensional) thread block to each
map in the previous layer and a thread to each neuron in every map. Similar to FP, for maps
with more than 1024 neurons, the 2D grid is further split into smaller 1D blocks by assigning a
2D block to each row of the map. Each thread computes the delta of its corresponding neuron by
pulling deltas from the current layer. For every neuron in the previous layer we have to determine
the list of neurons in the current layer which are connected to it. Let us consider neuron ( i;j)
from a map in layer Ln1, and then assume that ( x;y) are the coordinates of neurons in maps
ofLnthat contribute to the delta of neuron ( i;j). The (x;y) neuron is connected to kernel size
number neurons ( KxKy) from each connected map in the previous layer. The indices in Ln1
of the neurons connected through a kernel to the ( x;y) neuron are:
x(Sx+ 1) ix(Sx+ 1) +Kx1;
y(Sy+ 1) jy(Sy+ 1) +Ky1:
We present a fast, fully parameterizable GPU implementation of Convolutional
Neural Network variants. Our feature extractors are neither carefully designed
nor pre-wired, but rather learned in a supervised way. Our deep hierarchical
architectures achieve the best published results on benchmarks for object
classification (NORB, CIFAR10) and handwritten digit recognition (MNIST), with
error rates of 2.53%, 19.51%, 0.35%, respectively. Deep nets trained by simple
back-propagation perform better than more shallow ones. Learning is
surprisingly rapid. NORB is completely trained within five epochs. Test error
rates on MNIST drop to 2.42%, 0.97% and 0.48% after 1, 3 and 17 epochs,
respectively.
Dan C. Cireşan Ueli Meier Jonathan Masci Luca M. Gambardella Jürgen Schmidhuber