Effect of zeroth and first derivative on smoothing splines Let $\hat{f}=\arg min_f \left( \sum_{i=1}^n (Y_i -f(X_i))^2 + \lambda \int [f^{(m)}(x)]^2 dx\right)$
Suppose $\lambda = \infty$ and consider the cases when m = 0, m = 1. I am trying to understand what the subsequent function will look in both cases.
When m = 1, the penalizing term forces $\int [f’(x)]^2 dx$ to be small, so the slope of f(x) tends to 0, and thus f(x) will look like a step function. Is this line of reasoning correct?
When m = 0, the penalizing term forces $\int [f(x)]^2 dx$ to be small, thus the resulting function will simply be f(x) = 0?

As $\lambda \rightarrow \infty$, our smoothing penalty dominates the cost function in evaluating $f$'s associated costs. As a consequence $\hat{f}$ approximates the least squares solution we would get for $y \approx \beta_0 + \beta_1 x$, i.e. we get a straight line which has no "actual" second derivatives. Do note though that $f$ is defined to be a cubic spline based on the form presented here; so by definition, it is (at least) twice differentiable.
Given the above:

*

*for $m=1$ we won't have a step function because step functions are not valid smoothing functions. If any sharp inflexion points existed they would have infinite first derivatives as $\lambda$ got larger. In this scenario, as $\lambda$ gets larger, we have to predict the mean value $\bar{y}$ for all the values of $x$ because this prediction will allow us to get the smallest first derivatives possible (i.e. no slopes at any point) while minimising our RSS from the first term.

*for $m=0$ you are correct, we will indeed have $\hat{f}(x)\rightarrow0$, as even the "slightest fit" to the data would move us away from the minimum. In effect, we ignore the data. :D

