Answers.Body: <p>Your reference doesn't claim Laplace distribution is less sensitive to outliers than normal distribution, the intended claim is <em>the absolute error loss is more robust (less sensitive to outliers) than the squared error loss</em> used in many regression models.</p>
  <p>According to your quoted well-known duality between certain common loss functions and the maximum likelihood approach for regression problems in Bayesian statistics, and in particular the squared error loss is equivalent to the negative log-likelihood loss <span class="math-container">$J(\mathbf{\theta})$</span> (expressed on the same page) if we assume a same Gaussian distributed additive noise, and a detailed equivalence derivation is shown <a href="https://en.wikipedia.org/wiki/Maximum_likelihood_estimation#Continuous_distribution,_continuous_parameter_space" rel="nofollow noreferrer">here</a>. Similarly the absolute error loss corresponds to a Laplace distributed noise.</p>
  <p>Further per probability theory Laplace distribution generally has thicker tails compared to the Gaussian distribution, therefore Laplace distribution encodes sporadic larger noise values (as outliers) as more probable and thus possibly all likelihood factors <span class="math-container">$p(.)$</span> (between <span class="math-container">$0$</span> and <span class="math-container">$1$</span>) in the above <span class="math-container">$J(\mathbf{\theta})$</span> containing the same outlier data in the same training set would be larger in the Laplace noise case , given the parametric statistical model to be regressed. And due to negative log-likelihood loss formula, the positive <span class="math-container">$J(\mathbf{\theta})$</span> would actually be smaller after applying the 'negative log' functions. Therefore its dual which is the absolute error loss is also smaller in the sense that it's more robust (less sensitive to outliers) than the squared error loss.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-15T07:36:15.020Z
Answers.LastActivityDate: 2023-11-15T19:53:34.877Z
Answers.LastEditDate: 2023-11-15T19:53:34.877Z
Answers.UserDisplayName: cinch
Body: <p>Why is laplace distribution less sensitive to outliers than normal distribution?. The following is the content.</p>
  <p>(See page 103 of <a href="http://smlbook.org/book/sml-book-draft-latest.pdf" rel="nofollow noreferrer">http://smlbook.org/book/sml-book-draft-latest.pdf</a>)</p>
  <p>&quot;Also, in the regression case there is a duality between certain common loss functions and the maximum likelihood approach, as we have previously observed. For instance, in a regression model with additive noise, the squared error
  loss is equivalent to the negative log-likelihood if we assume a Gaussian noise distribution, <span class="math-container">$\epsilon \sim N(0, \sigma_{\epsilon}^2)$</span>. Similarly, we noted above that the absolute error loss
  corresponds to an implicit assumption of Laplace distributed noise,
  <span class="math-container">$\epsilon \sim \mathcal{L}(0, B_{\epsilon})$</span>.</p>
  <p>This statistical perspective is one way to understand the fact that the absolute error loss is more robust (less sensitive to outliers) than the squared error loss, since the Laplace distribution has thicker tails compared to the Gaussian distribution. The Laplace distribution therefore encodes sporadic large noise values (that is,
  outliers) as more probable, compared to the Gaussian distribution&quot;</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-11-14T01:37:21.053Z
Comments.Text: Can you provide more context? Where did you find this claim?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-11-14T08:06:53.820Z
Comments.Text: I updated the question with the content.
Comments.UserDisplayName: DSPinfinity
Tags: <machine-learning><statistics>
Title: Why is laplace distribution less sensitive to outliers than normal distribution?
UserDisplayName: DSPinfinity
ThreadText: # Why is laplace distribution less sensitive to outliers than normal distribution?
  
  (Asked by: username_0 on 2023-11-14)
  
  Why is laplace distribution less sensitive to outliers than normal distribution?. The following is the content.
  
  (See page 103 of <http://smlbook.org/book/sml-book-draft-latest.pdf>)
  
  "Also, in the regression case there is a duality between certain common loss functions and the maximum likelihood approach, as we have previously observed. For instance, in a regression model with additive noise, the squared error
  loss is equivalent to the negative log-likelihood if we assume a Gaussian noise distribution, $\epsilon \sim N(0, \sigma\_{\epsilon}^2)$. Similarly, we noted above that the absolute error loss
  corresponds to an implicit assumption of Laplace distributed noise,
  $\epsilon \sim \mathcal{L}(0, B\_{\epsilon})$.
  
  This statistical perspective is one way to understand the fact that the absolute error loss is more robust (less sensitive to outliers) than the squared error loss, since the Laplace distribution has thicker tails compared to the Gaussian distribution. The Laplace distribution therefore encodes sporadic large noise values (that is,
  outliers) as more probable, compared to the Gaussian distribution"
  
  **Tags:** machine-learning, statistics
  > **Comments:**
  > **username_1**: Can you provide more context? Where did you find this claim?  
  > **username_0**: I updated the question with the content.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-11-15. Score: 1
  
  Your reference doesn't claim Laplace distribution is less sensitive to outliers than normal distribution, the intended claim is *the absolute error loss is more robust (less sensitive to outliers) than the squared error loss* used in many regression models.
  
  According to your quoted well-known duality between certain common loss functions and the maximum likelihood approach for regression problems in Bayesian statistics, and in particular the squared error loss is equivalent to the negative log-likelihood loss $J(\mathbf{\theta})$ (expressed on the same page) if we assume a same Gaussian distributed additive noise, and a detailed equivalence derivation is shown [here](https://en.wikipedia.org/wiki/Maximum_likelihood_estimation#Continuous_distribution,_continuous_parameter_space). Similarly the absolute error loss corresponds to a Laplace distributed noise.
  
  Further per probability theory Laplace distribution generally has thicker tails compared to the Gaussian distribution, therefore Laplace distribution encodes sporadic larger noise values (as outliers) as more probable and thus possibly all likelihood factors $p(.)$ (between $0$ and $1$) in the above $J(\mathbf{\theta})$ containing the same outlier data in the same training set would be larger in the Laplace noise case , given the parametric statistical model to be regressed. And due to negative log-likelihood loss formula, the positive $J(\mathbf{\theta})$ would actually be smaller after applying the 'negative log' functions. Therefore its dual which is the absolute error loss is also smaller in the sense that it's more robust (less sensitive to outliers) than the squared error loss.
  
  
  ---
  
  