Answers.Body: <p>The <a href="https://en.wikipedia.org/wiki/Loss_function" rel="nofollow noreferrer">loss function</a> (aka cost function) measures the correctness of the predictions of the model. </p>
  
  <p>For example, a simple cost function could be <span class="math-container">$|y - f(x)|$</span>, where </p>
  
  <ul>
  <li><span class="math-container">$\hat{y} = f(x)$</span> is the prediction of the model <span class="math-container">$f$</span> when the input is <span class="math-container">$x$</span>,</li>
  <li><span class="math-container">$y$</span> is the ground-truth label for input <span class="math-container">$x$</span> (i.e. what the model is supposed to output when the input is <span class="math-container">$x$</span>), and</li>
  <li><span class="math-container">$|\cdot|$</span> is the absolute value</li>
  </ul>
  
  <p>The loss function is also used to understand whether a model is overfitting (and underfitting) or not. More specifically, if the loss of the model evaluated on the training data (the training loss) keeps decreasing while the loss of the model evaluated on the validation data (validation loss) starts to increase, that's a good sign that the model is overfitting (i.e. just memorizing the training data, but it will likely perform poorly on non-training data). If the training loss does not decrease, that's a good sign of underfitting (i.e. the model is not capable of learning the patterns in the training data), so you may need to use a model with a bigger capacity, change the loss, or do something else.</p>
  
  <p>The loss is typically not directly used as a measure of the <a href="https://stats.stackexchange.com/q/385312/82135">performance</a> of the model because the loss doesn't directly represent the performance of the model. In fact, the loss can be relatively big (with respect to another model's loss) and the model still <em>performs</em> well, but typically a lower loss corresponds to higher performance. The performance of the model is calculated differently depending on the task, model, and your goals. The most common performance measure is the <em>accuracy</em> (the number of correct predictions over the total number of predictions), but there are many other performance measures, such as the precision, recall, f1 score, AUC, etc., that emphasize different behavior that you expect from your model.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-14T19:49:02.860Z
Answers.LastActivityDate: 2020-06-14T20:01:28.423Z
Answers.LastEditDate: 2020-06-14T20:01:28.423Z
Answers.UserDisplayName: nbro
Body: <p>Why is it useful to track loss while the model is being trained?</p>
  
  <p>Options are:</p>
  
  <ol>
  <li>Loss is only useful as a final metric. It should not be evaluated while the model is being trained.</li>
  <li>Loss dictates how effective the model is.</li>
  <li>Loss can help understand how much the model is changing per iteration. When it converges, that's an indicator that further training will have little benefit.</li>
  <li>None of the above</li>
  </ol>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-14T15:05:41.843Z
Comments.Text: Hi and welcome to AI SE! Do you know something about machine learning or neural networks? This question should be easy to answer if you know something about neural networks and you have trained at least one once in your life.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-14T20:39:15.810Z
Comments.Text: This looks like coursework? If it is, could you please add some of your own thoughts/analysis (use [edit]). Also, you may want to consider whether you have agreed to any code of conduct on such a course - e.g. if your correct answer leads to gaining a certificate - because posting a question from course material directly in order to get someone else to answer for you may lead to your results being nullified.
Comments.UserDisplayName: Neil Slater
Tags: <machine-learning><objective-functions><homework>
Title: Why is it useful to track loss while model is being trained?
UserDisplayName: MILLIONISH
ThreadText: # Why is it useful to track loss while model is being trained?
  
  (Asked by: username_0 on 2020-06-14)
  
  Why is it useful to track loss while the model is being trained?
  
  Options are:
  
  1. Loss is only useful as a final metric. It should not be evaluated while the model is being trained.
  2. Loss dictates how effective the model is.
  3. Loss can help understand how much the model is changing per iteration. When it converges, that's an indicator that further training will have little benefit.
  4. None of the above
  
  **Tags:** machine-learning, objective-functions, homework
  
  > **Comments:**
  > **username_1**: Hi and welcome to AI SE! Do you know something about machine learning or neural networks? This question should be easy to answer if you know something about neural networks and you have trained at least one once in your life.  
  > **username_2**: This looks like coursework? If it is, could you please add some of your own thoughts/analysis (use [edit]). Also, you may want to consider whether you have agreed to any code of conduct on such a course - e.g. if your correct answer leads to gaining a certificate - because posting a question from course material directly in order to get someone else to answer for you may lead to your results being nullified.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-06-14. Score: 1
  
  The [loss function](https://en.wikipedia.org/wiki/Loss_function) (aka cost function) measures the correctness of the predictions of the model.
  
  For example, a simple cost function could be $|y - f(x)|$, where
  
  * $\hat{y} = f(x)$ is the prediction of the model $f$ when the input is $x$,
  * $y$ is the ground-truth label for input $x$ (i.e. what the model is supposed to output when the input is $x$), and
  * $|\cdot|$ is the absolute value
  
  The loss function is also used to understand whether a model is overfitting (and underfitting) or not. More specifically, if the loss of the model evaluated on the training data (the training loss) keeps decreasing while the loss of the model evaluated on the validation data (validation loss) starts to increase, that's a good sign that the model is overfitting (i.e. just memorizing the training data, but it will likely perform poorly on non-training data). If the training loss does not decrease, that's a good sign of underfitting (i.e. the model is not capable of learning the patterns in the training data), so you may need to use a model with a bigger capacity, change the loss, or do something else.
  
  The loss is typically not directly used as a measure of the [performance](https://stats.stackexchange.com/q/385312/82135) of the model because the loss doesn't directly represent the performance of the model. In fact, the loss can be relatively big (with respect to another model's loss) and the model still *performs* well, but typically a lower loss corresponds to higher performance. The performance of the model is calculated differently depending on the task, model, and your goals. The most common performance measure is the *accuracy* (the number of correct predictions over the total number of predictions), but there are many other performance measures, such as the precision, recall, f1 score, AUC, etc., that emphasize different behavior that you expect from your model.
  
  
  ---
  
  