Do Spline Models Have The Same Properties Of Standard Regression Models? This is a question I have been trying to look into.
Consider Linear Regression - depending on how the model parameters (i.e. "beta" regression coefficients) are estimated (e.g. Ordinary Least Squares, Maximum Likelihood Estimation), we know that the model will have certain statistical properties.
For example, we can prove that these estimates are statistically unbiased and consistent - this is said to be useful in various inference tasks that are associated with statistical analysis.
My question relates to whether similar properties are known to exist about Spline Regression Models. Conceptually, I have been told that we can think of Spline Models as "several mini regression models strategically placed together in a piece-wise manner". As a result, Spline Models are said to have a very high level of "flexibility" that in theory can allow them to capture complex relationships within the data better than traditional regression models - however, the tradeoff being that Spline Models are to said to often "overfit" the data if not implemented carefully.
But in terms of statistical properties of Spline Models - are they comparable to traditional regression models?
For example, suppose I fit a Spline Model to some data that comprises of 3 individual regression models that were smoothed and fit together - at the piecewise level (i.e. individual level), can we still construct Confidence Intervals for the parameter estimates of the Spline Model and make similar claims about their statistical properties?

If you’re comfortable with the usual assumptions (e.g., Gauss-Markov conditions), then yes.
Spline regressions just apply linear regression to spline-transformed variables. In many regards, that is no different from including a quadratic or a $\log$ as a feature, but instead you do a spline transformation.
Once you have your features, including any features you engineered through transformations, you just run linear regression on those numbers. The regression machinery does not know or care how you got those features.
I do have some doubts about some of the usual conditions holding, but I suppose missing-variable bias could always be a concern.
This video by MathematicalMonk (Jeffrey Miller) does a good job of explaining the idea of incorporating some kind of engineered “feature space” that combines the original features and transformations involving one or several features.
