Was Kernel Regression Invented to Address the Problems with Higher Order Polynomial Regressions? I had this thought today : We all know that higher order polynomials (e.g. polynomial regression models) have a tendency of overfitting the data and performing poorly (i.e. generalizing) to new data.
So: Was Kernel Regression used to to address this problem? It seems that "kernels" have the ability to take on complex shapes and patterns that are similar to higher order polynomial functions - but do not seem to be as notorious in terms of overfitting.
Is this the correct rationale? In part, were kernels (e.g. via kernel regression) used to address the problem of overfitting caused by higher order polynomials?
Thanks!

No, the kernel is simply an implicit expansion that (sometimes) makes things easier.
Take the RBF kernel, for example.
To expand the original variables in that basis you'd need an infinite amount of variables.
Due to this (often) higher dimensionality, kernel regression is usually paired with regularization (in the form of penalization of coefficients).
This penalization is often expressible in terms of the kernel.
This addresses the bias-variance tradeoff: higher penalization leads to higher bias and lower variance, giving you control of this scale.
The kernel is simply an efficient form of solving regression in fixed higher dimensions.
If you have a linear kernel and more data than variables ($n>p$) the kernel formulation is often less efficient than the direct one.
