On model selection and the disability of neural networks to decompose tasks
<|reference_start|>On model selection and the disability of neural networks to decompose tasks: A neural network with fixed topology can be regarded as a parametrization of functions, which decides on the correlations between functional variations when parameters are adapted. We propose an analysis, based on a differential geometry point of view, that allows to calculate these correlations. In practise, this describes how one response is unlearned while another is trained. Concerning conventional feed-forward neural networks we find that they generically introduce strong correlations, are predisposed to forgetting, and inappropriate for task decomposition. Perspectives to solve these problems are discussed.<|reference_end|>
@article{toussaint2002on,
    title={On model selection and the disability of neural networks to decompose
  tasks},
    author={Marc Toussaint},
    journal={Proceedings of the International Joint Conference on Neural
  Networks (IJCNN 2002), 245-250.},
    year={2002},
    archivePrefix={arXiv},
    eprint={nlin/0202038},
    primaryClass={nlin.AO cond-mat.dis-nn cs.NE}
}