  On-line and batch learning of a perceptron in a discrete weight space, where
each weight can take $2 L+1$ different values, are examined analytically and
numerically. The learning algorithm is based on the training of the continuous
perceptron and prediction following the clipped weights. The learning is
described by a new set of order parameters, composed of the overlaps between
the teacher and the continuous/clipped students. Different scenarios are
examined among them on-line learning with discrete/continuous transfer
functions and off-line Hebb learning. The generalization error of the clipped
weights decays asymptotically as $exp(-K \alpha^2)$/$exp(-e^{|\lambda|
\alpha})$ in the case of on-line learning with binary/continuous activation
functions, respectively, where $\alpha$ is the number of examples divided by N,
the size of the input vector and $K$ is a positive constant that decays
linearly with 1/L. For finite $N$ and $L$, a perfect agreement between the
discrete student and the teacher is obtained for $\alpha \propto \sqrt{L
\ln(NL)}$. A crossover to the generalization error $\propto 1/\alpha$,
characterized continuous weights with binary output, is obtained for synaptic
depth $L > O(\sqrt{N})$.

Statistical Mechanics