Answers.Body: <p>For the Logistic Regression Classifier and SVC, the <code>train_accuracy</code> and <code>test_accuracy</code> are very similar; thus, there is no evidence for over or under fitting.  However, the KNN, shows a <code>train_accuracy</code> that is lower than your <code>test_accuracy</code>, which <em>suggests</em> overfitting.  Alternatively, depending on your sample size, it is entirely possible that this variation is within the expected range of error.</p>
  <p>In machine learning, the model is trained on one dataset (i.e., the <strong>training data</strong>), but performing model evaluation (i.e., estimate &quot;real world&quot; performance by calculating <code>accuracy</code> or some other metric) on the same dataset that you trained on would be biased.  Instead, the evaluation is performed on an independent identically distributed dataset (i.e. the <strong>test data</strong>) to get an unbiased estimate of model performance.  However, it is still important to compare accuracy (or whatever performance metric you choose) between training and testing to assess for over- or underfitting.</p>
  <p><strong>Overfitting</strong> means that your model &quot;fit&quot; the training data very well and achieves high accuracy on that data.  However, the model performs poorly (or less well) on the test data.  This is because the model not only &quot;fits&quot; the signal in the training data but also the noise.  The problem is that noise is random and it will surely be different in the test set.  Therefore, the model doesn't generalize well to other datasets.</p>
  <p><strong>Underfitting</strong> means that the model performs poorly on both datasets.  This is typically because the model does not have the &quot;expressivity&quot; or &quot;capacity&quot; to learn the signals within the data or that the data is random and has no &quot;signal&quot; to speak of (and, therefore, nothing from which to learn).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-13T14:21:45.750Z
Answers.LastActivityDate: 2022-11-13T14:29:31.527Z
Answers.LastEditDate: 2022-11-13T14:29:31.527Z
Answers.UserDisplayName: Snehal Patel
Answers.Body: <p>From the roc_auc metric it does not show the model is in the overfit zone.</p>
  <p>It tells that on average of 10 predictions 8 predictions could be right and  2 predictions could go wrong.</p>
  <p>Now , it depends on the application domain to decide the accuracy you are expecting from the model</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-06T02:04:41.443Z
Answers.LastActivityDate: 2024-08-06T02:05:18.097Z
Answers.LastEditDate: 2024-08-06T02:05:18.097Z
Answers.UserDisplayName: Biku
Answers.Body: <p>Actually, in the strict sense, you cannot decide overfitting here, because overfitting is tested using losses, not metrics like accuracy.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-14T12:23:03.310Z
Answers.Text: For the downvote, why? This is correct :)
Answers.UserDisplayName: Dr. Snoopy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-05T15:00:18.143Z
Answers.Text: To my knowledge overfitting is a phenomenon that's unbound to any metric - it only means that your model is overly specific to the training data. You can identify an overfitted model with accuracy as well as with loss, no?
Answers.UserDisplayName: Chillston
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-05T15:46:48.890Z
Answers.Text: @Chillston no, the definition is to only use loss (since that is what the model is being trained on), not any metric, because metrics measure different things.
Answers.UserDisplayName: Dr. Snoopy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-05T17:46:29.707Z
Answers.Text: Interesting, do you have an intuition why this is defined so strictly? Where is the definition from? Not trying to be cheeky - just interested because its not super intuitive to me.
Answers.UserDisplayName: Chillston
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-14T08:40:39.820Z
Answers.LastActivityDate: 2022-11-14T08:40:39.820Z
Answers.UserDisplayName: Dr. Snoopy
Body: <p><a href="https://i.sstatic.net/uAwNX.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/uAwNX.png" alt="enter image description here" /></a></p>
  <p>What does this difference in train and test accuracy mean?</p>
  
Tags: <machine-learning><overfitting><performance><underfitting>
Title: How do you interpret this train vs test accuracy scores? is the model under or over fitting?
UserDisplayName: ProgrammingBot
ThreadText: # How do you interpret this train vs test accuracy scores? is the model under or over fitting?
  
  (Asked by: username_0 on 2022-11-13)
  
  [![enter image description here](https://i.sstatic.net/uAwNX.png)](https://i.sstatic.net/uAwNX.png)
  
  What does this difference in train and test accuracy mean?
  
  **Tags:** machine-learning, overfitting, performance, underfitting
  
  ---
  
  ## 3 answers
  
  ### Answer by username_1 on 2022-11-13. Score: 1
  
  For the Logistic Regression Classifier and SVC, the `train_accuracy` and `test_accuracy` are very similar; thus, there is no evidence for over or under fitting. However, the KNN, shows a `train_accuracy` that is lower than your `test_accuracy`, which *suggests* overfitting. Alternatively, depending on your sample size, it is entirely possible that this variation is within the expected range of error.
  
  In machine learning, the model is trained on one dataset (i.e., the **training data**), but performing model evaluation (i.e., estimate "real world" performance by calculating `accuracy` or some other metric) on the same dataset that you trained on would be biased. Instead, the evaluation is performed on an independent identically distributed dataset (i.e. the **test data**) to get an unbiased estimate of model performance. However, it is still important to compare accuracy (or whatever performance metric you choose) between training and testing to assess for over- or underfitting.
  
  **Overfitting** means that your model "fit" the training data very well and achieves high accuracy on that data. However, the model performs poorly (or less well) on the test data. This is because the model not only "fits" the signal in the training data but also the noise. The problem is that noise is random and it will surely be different in the test set. Therefore, the model doesn't generalize well to other datasets.
  
  **Underfitting** means that the model performs poorly on both datasets. This is typically because the model does not have the "expressivity" or "capacity" to learn the signals within the data or that the data is random and has no "signal" to speak of (and, therefore, nothing from which to learn).
  
  
  ---
  
  ### Answer by username_2 on 2024-08-06. Score: 0
  
  From the roc\_auc metric it does not show the model is in the overfit zone.
  
  It tells that on average of 10 predictions 8 predictions could be right and 2 predictions could go wrong.
  
  Now , it depends on the application domain to decide the accuracy you are expecting from the model
  
  
  ---
  
  ### Answer by username_3 on 2022-11-14. Score: -1
  
  Actually, in the strict sense, you cannot decide overfitting here, because overfitting is tested using losses, not metrics like accuracy.
  
  > **Comments:**
  > **username_3**: For the downvote, why? This is correct :)  
  > **username_4**: To my knowledge overfitting is a phenomenon that's unbound to any metric - it only means that your model is overly specific to the training data. You can identify an overfitted model with accuracy as well as with loss, no?  
  > **username_3**: @Chillston no, the definition is to only use loss (since that is what the model is being trained on), not any metric, because metrics measure different things.  
  > **username_4**: Interesting, do you have an intuition why this is defined so strictly? Where is the definition from? Not trying to be cheeky - just interested because its not super intuitive to me.  
  
  ---
  
  