This paper focuses on multilayer perceptron neural networks where the activation functions are adaptive and where each neuron synapses is modelled by a finite impulse response (FIR) filter. A simplified architecture consisting of a variable activation (VA) function which is sandwiched between two FIR synapses is studied. The VA function consists of, a mixed linear-tank sigmoid with a parameter which controls the linear region. The VA parameters and FIR synaptic weights are updated using a modified form of the instantaneous-cost (IC) temporal backpropagation algorithm. Simulations for identifying cascaded nonlinear transfer functions with internal memory and arbitrary activation functions illustrate the improved modelling performance over models with non-adaptive activation functions.
['A.N. Birkett', 'R.A. Goubran']
Nonlinear adaptive filtering with FIR synapses and adaptive activation functions