When will L2 norm and squared L2 norm be equivalent in ridge regression? I am doing ridge regression and am wondering why it uses squared L2 norm. This post Why l2 norm squared but l1 norm not squared? gave some thought about why using it. But a further question arises: when will using L2 and squared L2 norm be equivalent?
In my case, I am dealing with a completely orthonormal case. It is a cell-means model with the objective function in the form $\sum_i\sum_j(X_{ij}-\mu_i)^2$:

*

*Using squared L2 norm adds a term $\lambda(\mu_1^2+\cdots+\mu_p^2) $ to the objective function, which then reduces to p separate univariate optimizations.

*Using L2 norm adds a term $\lambda\sqrt{(\mu_1^2+\cdots+\mu_p^2)} $, which cannot be reduced to univariate optimizations because of the square root of sum of all parameters.

In this case, I don't think those two are still equivalent. So which one to use?
(I think from the KKT condition we should conclude that those two are equivalent, so is there something in the orthonormal case that violates some assumptions?)

The definition of ridge regression uses the squared $L2$ norm for the penalty. The objective function is
$$L(\beta; \lambda) = \sum_{i = 1}^{n}(y_i - x_i\beta)^2 + \lambda \sum_{j = 1}^{p} \beta_j^2 \space \space,$$
where $p = \dim(\beta)$.
In that sense, your question has an easy answer: Ridge regression uses the squared $L2$ norm because that's the defition of ridge regression.
If all you care about is getting the correct coefficients, it doesn't matter if you use the $L2$ norm or its square. Compare the two objective functions term by term:
$$\sum_{i = 1}^{n}(y_i - x_i\beta)^2 + \lambda \sum_{j = 1}^{p} \beta_j^2 \tag{1}$$
$$\sum_{i = 1}^{n}(y_i - x_i\beta)^2 + \lambda \sqrt{\sum_{j = 1}^{p} \beta_j^2} \tag{2}$$
Obviously, the $\sum_{i = 1}^{n}(y_i - x_i\beta)^2$ term is nonnegative and maximized at the same point in $(1)$ and $(2)$. The penalty terms, $\sum_{j = 1}^{p} \beta_j^2$ and $\sqrt{\sum_{j = 1}^{p} \beta_j^2}$, are also maximized at the same value. Both penalties are non-negative, so taking the square (or square root) of one doesn't change the value where it reaches its optimum. The value of the tuning parameter, $\lambda$ will be different, but that doesn't change the optimum.
Edit: @Zhanxiong pointed out this doesn't show that the optimum, $\hat{\beta}$, will be the same when both parts of $(1)$ and $(2)$ are combined. As it stands, the argument above is more of a heuristic than a proof.
Like you said, the squared $L2$ norm is differentiable while the $L2$ norm is not. Since you get the same answer, either way, you might as well maximize a function that's differentiable.
