Does it make sense to use neural network in place of polynomial regression to fit 1 dimensional functions when only 1 feature is available? As title says I would like to understand why there are so many tutorials and notebooks showing how to fit 1 dimensional functions  with neural networks instead of polynomial regression. What are the advantages of using MLPs? do they generalize better? 1re they more powerful function approximators?

Neural networks are not all that special. Yes, there are these universal approximation theorems saying that, given a decent function, a neural network can approximate it as close as you require. This sounds great until you realize:

*

*The Stone-Weierstrass theorem says about the same for polynomial approximations


*Carleson’s theorem says about the same for approximating functions using Fourier series


*No universal approximation theorem (and neither Stone-Weierstrass nor Carleson) says how neural networks perform in the presence of noise (the regression error term).
Consequently, neural networks could be considered somewhat overrated (and I say this as someone who likes them and thinks they’re cool).
Cynically, I think people are a bit mesmerized by neural networks. They have seen specific architectures like convolutional neural networks have great success at image recognition, and they want those super-high accuracy scores for their own problems. That makes training in neural networks to be in demand, so tutorial-makers give the people what they want.
Polynomial regression as an alternative to neural nets is a provocative paper that is worth reading.
Cheng, Xi, et al. "Polynomial regression as an alternative to neural nets." arXiv preprint arXiv:1806.06850 (2018).
