How does ridge regression reduce the variance of the estimates of $\beta$ In the scikit-learn library, Ridge class, there is a note that reads: "Regularization improves the conditioning of the problem and reduces the variance of the estimates."
Given the expression of beta for ridge regression:
$\beta = (X^T X + \lambda I_d)^{-1} X^T y$
How does the addition of $\lambda I_d$ to the moments matrix improve the condition number of the inverse? And how exactly does it reduce the variance of the estimates (mathematically)?

The additional term $\lambda I_d$ can be interpreted as adding some bias and it is the bias that reduces the variance.

*

*The addition of $\lambda I_d$ is equivalent to adding a penalty term to the least squares optimization problem $$\min_{\beta}: \overbrace{ (y-X\beta )^T(y-X\beta ) }^{\text{least squares}}+ \overbrace{\lambda (\beta^T\beta)}^{\text{additional penalty}}$$


*The penalty term is equivalent to constraining the parameters to a particular maximum size.
$$\min_{\beta}: { (y-X\beta )^T(y-X\beta ) } \qquad \text{and} \qquad \beta^T\beta < c$$


*Constraining the
parameters will reduce the variance of the parameters.
