Approximation and Gradient Descent Training with Neural Networks
<|reference_start|>Approximation and Gradient Descent Training with Neural Networks: It is well understood that neural networks with carefully hand-picked weights provide powerful function approximation and that they can be successfully trained in over-parametrized regimes. Since over-parametrization ensures zero training error, these two theories are not immediately compatible. Recent work uses the smoothness that is required for approximation results to extend a neural tangent kernel (NTK) optimization argument to an under-parametrized regime and show direct approximation bounds for networks trained by gradient flow. Since gradient flow is only an idealization of a practical method, this paper establishes analogous results for networks trained by gradient descent.<|reference_end|>
@article{welper2024approximation,
    title={Approximation and Gradient Descent Training with Neural Networks},
    author={G. Welper},
    journal={arXiv preprint arXiv:2405.11696},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.11696},
    primaryClass={cs.LG}
}