  The paper proposes the use of structured neural networks for reinforcement
learning based nonlinear adaptive control. The focus is on partially observable
systems, with separate neural networks for the state and feedforward observer
and the state feedback and feedforward controller. The observer dynamics are
modelled by recurrent neural networks while a standard network is used for the
controller. As discussed in the paper, this leads to a separation of the
observer dynamics to the recurrent neural network part, and the state feedback
to the feedback and feedforward network. The structured approach reduces the
computational complexity and gives the reinforcement learning based controller
an {\em understandable} structure as compared to when one single neural network
is used. As shown by simulation the proposed structure has the additional and
main advantage that the training becomes significantly faster. Two ways to
include feedforward structure are presented, one related to state feedback
control and one related to classical feedforward control. The latter method
introduces further structure with a separate recurrent neural network that
processes only the measured disturbance. When evaluated with simulation on a
nonlinear cascaded double tank process, the method with most structure performs
the best, with excellent feedforward disturbance rejection gains.

Systems and Control;Machine Learning;Systems and Control