Answers.Body: <p>The following graph shows the constraint region (green), along with contours for Residual sum of squares (red ellipse). These are iso-lines signifying that points on an ellipse have the same RSS.
  <a href="https://i.sstatic.net/0VhAZ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/0VhAZ.png" alt="enter image description here" /></a>
  <em><strong>Figure</strong>: Lasso (left) and Ridge (right) Constraints</em>  [Source: Elements of Statistical Learning]</p>
  <p>As <strong>Ridge</strong> regression has a circular constraint (<span class="math-container">$\beta_1^2 + \beta_2^2 &lt;= d$</span>) with no edges, the intersection will not occur on an axis, signifying that the ridge regression parameters will usually be  non-zero.</p>
  <p>On the contrary, the <strong>Lasso</strong> constraint (<span class="math-container">$|\beta_1| + |\beta_2| &lt;= d$</span>) has corners at each of the axes, and so the ellipse will often intersect the constraint region at an axis. In 2D, such a scenario would result in one of the parameters to become zero whereas in higher dimensions, more of the parameter estimates may simultaneously reach zero.</p>
  <p>This is a <em>disadvantage</em> of ridge regression, wherein the least important predictors never get eliminated, resulting in the final model to contain all predictor variables. For Lasso, the L1 penalty forces some parameters to be exactly equal to zero when <span class="math-container">$\lambda$</span> is large. This has a dimensionality reduction effect resulting in sparse models.</p>
  <p>In cases where the number of predictors are small, L2 could be chosen over L1 as it constraints the coefficient norm retaining all predictor variables.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-07T20:59:22.287Z
Answers.Text: Your answer describes L1 and L2 regularization and their effect, but how does this relate to the concept of regularization? You just clarify what a person means by "regularization" and in which cases is L1 preferred over L2 (and vice versa), which is the actual question (I think).
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-07T21:26:31.747Z
Answers.Text: Given that the author of the question asked for a graphical intuition behind regularization (with the constraint curve), I am given to understand he has knowledge of what "Regularization" signifies. The answer simply does not warrant a definition of the term.
Answers.UserDisplayName: s_bh
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-07T20:50:56.980Z
Answers.LastActivityDate: 2020-07-24T10:57:48.243Z
Answers.LastEditDate: 2020-07-24T10:57:48.243Z
Answers.UserDisplayName: s_bh
Body: <p>Given a ridge and a lasso regularizer, which one should be chosen for better performance? </p>
  
  <p>An intuitive graphical explanation (intersection of the elliptical contours of the loss function with the region of constraints) would be helpful.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-07T20:15:39.813Z
Comments.Text: L1 and L2 have slightly different effects. Have you read anything about them?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-07T20:19:40.540Z
Comments.Text: Yes, I do have an understanding in terms of the math involved. I am interested to know about the effect that it has on the parameters, that help in achieving the regularization effect.
Comments.UserDisplayName: jaeger6
Tags: <machine-learning><comparison><regularization><l2-regularization><l1-regularization>
Title: Which is a better form of regularization: lasso (L1) or ridge (L2)?
UserDisplayName: jaeger6
ThreadText: # Which is a better form of regularization: lasso (L1) or ridge (L2)?
  
  (Asked by: username_0 on 2020-02-07)
  
  Given a ridge and a lasso regularizer, which one should be chosen for better performance?
  
  An intuitive graphical explanation (intersection of the elliptical contours of the loss function with the region of constraints) would be helpful.
  
  **Tags:** machine-learning, comparison, regularization, l2-regularization, l1-regularization
  > **Comments:**
  > **username_1**: L1 and L2 have slightly different effects. Have you read anything about them?  
  > **username_0**: Yes, I do have an understanding in terms of the math involved. I am interested to know about the effect that it has on the parameters, that help in achieving the regularization effect.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-02-07. Score: 3 | Accepted answer
  
  The following graph shows the constraint region (green), along with contours for Residual sum of squares (red ellipse). These are iso-lines signifying that points on an ellipse have the same RSS.
  [![enter image description here](https://i.sstatic.net/0VhAZ.png)](https://i.sstatic.net/0VhAZ.png)
  ***Figure**: Lasso (left) and Ridge (right) Constraints* [Source: Elements of Statistical Learning]
  
  As **Ridge** regression has a circular constraint ($\beta\_1^2 + \beta\_2^2 <= d$) with no edges, the intersection will not occur on an axis, signifying that the ridge regression parameters will usually be non-zero.
  
  On the contrary, the **Lasso** constraint ($|\beta\_1| + |\beta\_2| <= d$) has corners at each of the axes, and so the ellipse will often intersect the constraint region at an axis. In 2D, such a scenario would result in one of the parameters to become zero whereas in higher dimensions, more of the parameter estimates may simultaneously reach zero.
  
  This is a *disadvantage* of ridge regression, wherein the least important predictors never get eliminated, resulting in the final model to contain all predictor variables. For Lasso, the L1 penalty forces some parameters to be exactly equal to zero when $\lambda$ is large. This has a dimensionality reduction effect resulting in sparse models.
  
  In cases where the number of predictors are small, L2 could be chosen over L1 as it constraints the coefficient norm retaining all predictor variables.
  
  > **Comments:**
  > **username_1**: Your answer describes L1 and L2 regularization and their effect, but how does this relate to the concept of regularization? You just clarify what a person means by "regularization" and in which cases is L1 preferred over L2 (and vice versa), which is the actual question (I think).  
  > **username_2**: Given that the author of the question asked for a graphical intuition behind regularization (with the constraint curve), I am given to understand he has knowledge of what "Regularization" signifies. The answer simply does not warrant a definition of the term.  
  
  ---
  
  