Artificial neural networks have utility in a wide variety of computing environments, such as speech recognition, process control, optical character recognition, signal processing, and image processing. Processing engines for many of the foregoing computing environments may be implemented through neural networks comprising a plurality of elemental logic elements called neuron circuits.
A neuron circuit (or processing element) is the fundamental building block of a neural network. A neuron circuit has multiple inputs and one output. The structure of a conventional neuron circuit often includes a multiplier circuit, a summing circuit, a circuit for performing a non-linear function (such as a binary threshold or sigmoid function), and circuitry functioning as synapses or weighted input connections. Refer to FIG. 1, wherein inputs x.sub.1 -x.sub.n are weighted by respective synapses w.sub.1 -w.sub.n and accumulated together by summing circuit 2. The output of summing circuit 2 is fed into non-linear circuit 4 to generate the neuron circuit output 5.
FIG. 2 shows a non-linear transfer function in the form of a sigmoid-shaped function which is used by the prior art neuron circuit shown in FIG. 1. In the example shown, the sigmoid curve 6 is expressed by the equation: EQU OUTPUT=1/(1+e.sup.-NET) EQUATION 1
FIG. 3 shows another prior art neuron circuit, referred to as a perceptron neuron, which employs a binary threshold function. In this example, the perceptron neuron uses a binary threshold 14 as the non-linear function.
In summary, a typical conventional neuron circuit requires circuitry for weighted input connections, a summing circuit, a multiplier circuit, and complex circuitry for performing the non-linear function. Thus, the number of conventional neuron circuits which can be manufactured on a semiconductor chip is severely limited.
Therefore there is a significant need for a neuron circuit which has a minimum of components and which is simple and inexpensive to implement.
Conventional neural networks built of prior art neuron circuits require very lengthy training cycles, and even then they usually fail to converge on the correct result for every possible combination of input values. This is referred to in the art as achieving only a "local minimum" rather than a "global solution".
For example, the Feb. 18, 1993 issue of Electronic Design, p. 51, states that approximately 3 trillion (3.times.10.sup.12) training operations are required to train a typical neural network utilizing prior art neuron circuits. This typically requires weeks or even months of computational time, often using a super-computer.
Therefore, there is also a significant need for an artificial neuron that can form the basis of a neural network which does not require lengthy training cycles and which converges on a global solution in a single training cycle.