A biological neuron is a single nerve cell responsive to stimuli through weighted inputs known as synapses. The weighted stimuli are summed and processed through a particular non-linearity associated with the neuron. The output of the neuron may then be connected to the synapses of other neurons forming an interconnection of neurons known as a neural network, the latter of which possesses certain desirable properties including the ability to learn and recognize information patterns in a parallel manner.
Technologists have long studied the advantageous nature of the biological neuron in an attempt to emulate its behavior in electronic circuitry. As indicated, some of the advantages of the neural network include the learning and recognition of patterns and shapes. The neural network may be taught a particular pattern and later be called upon to identify the pattern from a distorted facsimile of the same pattern. Unlike conventional signal processing techniques wherein the solution is programmed with a predetermined algorithm, the recognition technique of the neural network may be learned through an iterative process of applying repeated samples of an input signal, comparing the output signal thereof with the desired response and adjusting the synaptic weights until the desired response is achieved.
Another advantageous feature of the neural network is the inherent parallel processing offering real-time execution without the classic von Neumann bottleneck associated with conventional serial processing techniques, i.e., digital signal processing and general microprocessor based applications. In general, neural networks are well suited for applications wherein the solution algorithm is not known or difficult to implement, or where conventional processing techniques are too slow. Notably, neural networks typically are not high precision devices, thus, a certain degree of error is generally present in the output signal.
Conventional neural networks typically comprise a predetermined, often rigid structure, wherein the network is designed with a particular application in mind, i.e., visual pattern recognition or speech recognition. Such neural networks work well for learning and recognizing static patterns to provide a higher-level output signal as compared to the input signal thereof, but usually fail to provide satisfactory results when confronted with a spatial input signal operating in a temporal environment. Furthermore, conventional neural networks typically require special organizational steps and new training sets for each new application. The need for the predefined, highly specialized neural structure and initialization data sets limits the versatility of the neural network. A modular approach would be more desirable wherein a single-level neural field could be designed with some functional flexibility whereby several modules may be easily interconnected to perform a variety of learning and recognition operations. Moreover, it would also be desirable for the neural network to re-synthesize the input signal from the output signal thereby recreating the original lower-level information.
Hence, there is a need for a hierarchical, self-organizing neural network for use in general purpose, bi-directional recognition and synthesis processes wherein a first input signal having a low-level of information content is recognized for providing a first output signal having a comparatively high-level of information content while, in the opposite direction, a second low-level output signal is synthesized from a second high-level input signal.