Answers.Body: <p>This is the sum of squared residuals, and it uses notation from the mathematical subfield of linear algebra (arguably functional analysis).</p>
  <p>The double vertical bars indicate that we use the mathematical operation of a <a href="https://en.wikipedia.org/wiki/Norm_(mathematics)" rel="nofollow noreferrer">norm</a>, which is a special kind of measure of distance that holds familiar properties.</p>
  <p>The subscript <span class="math-container">$2$</span> means that it is the <span class="math-container">$L_2$</span> norm. Until you know this notation, there’s no way to know that, but it’s so common that it’s worth remembering what an <span class="math-container">$L_p$</span> norm means. Let <span class="math-container">$x=(x_1,\dots,x_n)\in\mathbb R^n$</span> be a vector of real numbers.</p>
  <p><span class="math-container">$$
  \vert \vert
  x 
  \vert\vert_p
  =
  \left(
  \sum_{i=1}^n\vert x_i\vert^p
  \right)^{1/p}
  $$</span></p>
  <p>For the <span class="math-container">$2$</span>-norm in your equation, this is just the usual Euclidean distance.</p>
  <p>The <span class="math-container">$2$</span> superscript has its usual meaning, that you square the <span class="math-container">$L_2$</span> norm once you calculate it.</p>
  <p>Finally, <span class="math-container">$y-\hat y$</span> means that you take the vector of true <span class="math-container">$y$</span>-values and subtract the vector of predicted <span class="math-container">$y$</span>-values, denoted by <span class="math-container">$\hat y$</span>.</p>
  <p>In total:</p>
  <ol>
  <li><p>Take the difference between the true and predicted values.</p>
  </li>
  <li><p>Square each component of that difference.</p>
  </li>
  <li><p>Add up all of those squared differences.</p>
  </li>
  <li><p>Take the square root of that sum.</p>
  </li>
  <li><p>Square that square root.</p>
  </li>
  </ol>
  <p>As the last step undoes the fourth step to get you back to the third step, you can stop at step <span class="math-container">$3$</span>.</p>
  <p><strong>EDIT</strong></p>
  <p>It is common to refer to the residuals as errors. From a statistical standpoint, this is not quite correct, though it seems to be a reasonable slang that does not cause confusion among experienced practitioners.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-12T01:02:13.223Z
Answers.LastActivityDate: 2022-08-12T01:09:14.570Z
Answers.LastEditDate: 2022-08-12T01:09:14.570Z
Answers.UserDisplayName: Dave
Body: <p>In this <a href="https://www.deeplearning.ai/ai-notes/optimization/" rel="nofollow noreferrer">AI note</a> from <a href="https://deeplearning.ai" rel="nofollow noreferrer">https://deeplearning.ai</a>, the loss function below is used for a regression problem. However, I don't know how to interpret this loss function.</p>
  <p><a href="https://i.sstatic.net/28SBx.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/28SBx.png" alt="loss function" /></a></p>
  <p>First, does the author take the square of the difference between y-hat (prediction) and y (ground truth), so that positive and negative numbers don't cancel each other out? If so, why do we take the norm or distance, as well? Isn't the norm positive anyway? Or does he take the square so that it's more convenient to calculate the derivative?</p>
  <p>Second, what does the other 2 in subscript mean? It's not explained in the note and I was also not able to derive it from the context. All I know is that y ∈ R.</p>
  <p>It seems like I find it difficult to read mathematical notations. If you know a resource that explains these notations, please let me know.</p>
  
Tags: <regression><loss>
Title: How do I interpret this loss function?
UserDisplayName: Zarif
ThreadText: # How do I interpret this loss function?
  
  (Asked by: username_0 on 2022-08-11)
  
  In this [AI note](https://www.deeplearning.ai/ai-notes/optimization/) from <https://deeplearning.ai>, the loss function below is used for a regression problem. However, I don't know how to interpret this loss function.
  
  [![loss function](https://i.sstatic.net/28SBx.png)](https://i.sstatic.net/28SBx.png)
  
  First, does the author take the square of the difference between y-hat (prediction) and y (ground truth), so that positive and negative numbers don't cancel each other out? If so, why do we take the norm or distance, as well? Isn't the norm positive anyway? Or does he take the square so that it's more convenient to calculate the derivative?
  
  Second, what does the other 2 in subscript mean? It's not explained in the note and I was also not able to derive it from the context. All I know is that y ∈ R.
  
  It seems like I find it difficult to read mathematical notations. If you know a resource that explains these notations, please let me know.
  
  **Tags:** regression, loss
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-08-12. Score: 4 | Accepted answer
  
  This is the sum of squared residuals, and it uses notation from the mathematical subfield of linear algebra (arguably functional analysis).
  
  The double vertical bars indicate that we use the mathematical operation of a [norm](https://en.wikipedia.org/wiki/Norm_(mathematics)), which is a special kind of measure of distance that holds familiar properties.
  
  The subscript $2$ means that it is the $L\_2$ norm. Until you know this notation, there’s no way to know that, but it’s so common that it’s worth remembering what an $L\_p$ norm means. Let $x=(x\_1,\dots,x\_n)\in\mathbb R^n$ be a vector of real numbers.
  
  $$
  \vert \vert
  x
  \vert\vert\_p
  =
  \left(
  \sum\_{i=1}^n\vert x\_i\vert^p
  \right)^{1/p}
  $$
  
  For the $2$-norm in your equation, this is just the usual Euclidean distance.
  
  The $2$ superscript has its usual meaning, that you square the $L\_2$ norm once you calculate it.
  
  Finally, $y-\hat y$ means that you take the vector of true $y$-values and subtract the vector of predicted $y$-values, denoted by $\hat y$.
  
  In total:
  
  1. Take the difference between the true and predicted values.
  2. Square each component of that difference.
  3. Add up all of those squared differences.
  4. Take the square root of that sum.
  5. Square that square root.
  
  As the last step undoes the fourth step to get you back to the third step, you can stop at step $3$.
  
  **EDIT**
  
  It is common to refer to the residuals as errors. From a statistical standpoint, this is not quite correct, though it seems to be a reasonable slang that does not cause confusion among experienced practitioners.
  
  
  ---
  
  