Why l2 norm squared but l1 norm not squared? In the Lasso, and ElasticNet, we use, as penalty, the l1 norm without squaring. But in the ElasticNet and Ridge, we use the l2 norm squared. Why is that, is there a particular reason (computational, optimization dynamics, statistical?)
Because from a purely constrained/penalized convex optimization perspective, looks like both (squared or not squared l1 norm) would solve a similar problem (since $\|x\|_1<D \iff \|x\|_1^2<D^2$), so I guess choosing either l1 squared or plain l1 would only impact the training dynamics (is that correct) ? Could it be then because for non-smooth problems (like with $\ell_1$), one may want to have Lipschitz continuity (which is the case for non-squared l1 but not for squared l1) since we do not have smoothness ?
Curious about this (would be glad to read references).


But in the ElasticNet and Ridge, we use the l2 norm squared. Why is that, is there a particular reason (computational, optimization dynamics, statistical?)

A possible reason for the l2 norm being squared in ridge regression (or Tikhonov regularisation) is that it allows an easy expression for the solution of the problem $$\hat\beta = (\textbf{X}^T\textbf{X} + \lambda \textbf{I})^{-1} \textbf{X}^T y$$ where $\textbf{X}$ is the regressor matrix or design matrix, $\lambda$ the scaling parameter for the penalty, $\textbf{I}$ the identity matrix, $y$ the observations, and $\hat{\beta}$ the estimate of the coefficients.

That solution can be derived by taking the derivative of the cost function and setting it equal to zero
$$\nabla_{\hat\beta} \left[  (y - \textbf{X}\hat\beta )^T(y - \textbf{X}\hat\beta ) +  \hat\beta^T  \lambda \textbf{I} \hat\beta \right] = \textbf{X}^T \left (y - \textbf{X}\hat\beta \right)+  \lambda \textbf{I} \hat\beta  = \textbf{0}$$
