Answers.Body: <p>To evaluate your fit, the results on the validation set, as you say, are essential.</p>
  <ul>
  <li>A good performance on both the training and the validation set is a sign of a good fit</li>
  <li>A poor performance on the training set (and therefore on the validations set) is a sign of <em>underfitting</em></li>
  <li>A good performance on the training set, coupled with a poor performance on the validation set is a sign of <em>overfitting</em>.</li>
  </ul>
  <p>As you can see you cannot evaluate based on the training set alone, whether it's a good fit or an overfit of the training set.</p>
  <p>However, you stated in a comment that you also got 98% on the validation set. We can therefore conclude that you do have a good fit and a working model.</p>
  <p>The very high resulting score is probably due to the fact that you're working on a very <em>easy task</em>. The size shown in the image seems to be 100 samples per class -- this makes me think this is a 'toy' dataset used just as an exercise. Whether this works in the real world, whatever your objective is, depends on how representative this dataset is.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-06T18:05:19.317Z
Answers.Text: I see what you mean. The reason there's only 100 samples is because I split my dataset to be 80% for training and 20% for testing. Would increasing the testing split make it make it more obvious?
  
  I changed the split to be 50/50 just to see the difference. The new report has around 500 samples. It reported 100 training score and 93 cv score.
Answers.UserDisplayName: Abrahim Mahmud
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-07T07:59:46.363Z
Answers.Text: The exact size doesn't matter as long as they are both statistically representative, and 80% training is kind of standard. But this isn't the point, what I mean is this is an "easy" task for the model to solve.
Answers.UserDisplayName: Martino
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-06T10:58:17.403Z
Answers.LastActivityDate: 2022-07-06T10:58:17.403Z
Answers.UserDisplayName: Martino
Body: <p>I was playing around with some models and training them on a heart disease dataset and found my Extra Tree Classifier reported a 100% accuracy. I was shocked at first since I had never gotten a perfect score report from a model, but I realized that the model is most likely over/underfit. After doing some research, I found that you can determine if a model is over/underfit by comparing the Training Score and Cross Validation Score reports, but after looking at the report, I could not tell if my model was over/underfit or if it really is perfect. (I am new to machine learning and I have high doubts that it is actually perfect.)
  Here is the model report:</p>
  <p><a href="https://i.sstatic.net/bTJMU.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/bTJMU.png" alt="Extra Tree Classifier Model Report" /></a></p>
  <p>Can someone help me figure out if this model is over/underfit? (If I need to give any more data/info to figure this out, please let me know!)</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-07-06T08:16:02.530Z
Comments.Text: "you can determine if a model is over/underfit by comparing the Training Score and Cross Validation Score reports" > did you do that? How's the performance on the validation set?
Comments.UserDisplayName: Martino
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-07-06T09:17:02.860Z
Comments.Text: Based off the report, the training score was 100 and the cross validation score was about 98.
Comments.UserDisplayName: Abrahim Mahmud
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-07-08T03:35:20.463Z
Comments.Text: Turns out the model contained a whole lot of bias because 72% of the dataset was duplicated data. The reason for the CV and training score being so close is because the testing data was basically the same as the training data.
Comments.UserDisplayName: Abrahim Mahmud
Tags: <machine-learning><classification><python>
Title: Is My Extra Tree Classifier Model Over/Underfit?
UserDisplayName: Abrahim Mahmud
ThreadText: # Is My Extra Tree Classifier Model Over/Underfit?
  
  (Asked by: username_0 on 2022-07-05)
  
  I was playing around with some models and training them on a heart disease dataset and found my Extra Tree Classifier reported a 100% accuracy. I was shocked at first since I had never gotten a perfect score report from a model, but I realized that the model is most likely over/underfit. After doing some research, I found that you can determine if a model is over/underfit by comparing the Training Score and Cross Validation Score reports, but after looking at the report, I could not tell if my model was over/underfit or if it really is perfect. (I am new to machine learning and I have high doubts that it is actually perfect.)
  Here is the model report:
  
  [![Extra Tree Classifier Model Report](https://i.sstatic.net/bTJMU.png)](https://i.sstatic.net/bTJMU.png)
  
  Can someone help me figure out if this model is over/underfit? (If I need to give any more data/info to figure this out, please let me know!)
  
  **Tags:** machine-learning, classification, python
  
  > **Comments:**
  > **username_1**: "you can determine if a model is over/underfit by comparing the Training Score and Cross Validation Score reports" > did you do that? How's the performance on the validation set?  
  > **username_0**: Based off the report, the training score was 100 and the cross validation score was about 98.  
  > **username_0**: Turns out the model contained a whole lot of bias because 72% of the dataset was duplicated data. The reason for the CV and training score being so close is because the testing data was basically the same as the training data.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-07-06. Score: 0 | Accepted answer
  
  To evaluate your fit, the results on the validation set, as you say, are essential.
  
  * A good performance on both the training and the validation set is a sign of a good fit
  * A poor performance on the training set (and therefore on the validations set) is a sign of *underfitting*
  * A good performance on the training set, coupled with a poor performance on the validation set is a sign of *overfitting*.
  
  As you can see you cannot evaluate based on the training set alone, whether it's a good fit or an overfit of the training set.
  
  However, you stated in a comment that you also got 98% on the validation set. We can therefore conclude that you do have a good fit and a working model.
  
  The very high resulting score is probably due to the fact that you're working on a very *easy task*. The size shown in the image seems to be 100 samples per class -- this makes me think this is a 'toy' dataset used just as an exercise. Whether this works in the real world, whatever your objective is, depends on how representative this dataset is.
  
  > **Comments:**
  > **username_0**: I see what you mean. The reason there's only 100 samples is because I split my dataset to be 80% for training and 20% for testing. Would increasing the testing split make it make it more obvious?  I changed the split to be 50/50 just to see the difference. The new report has around 500 samples. It reported 100 training score and 93 cv score.  
  > **username_1**: The exact size doesn't matter as long as they are both statistically representative, and 80% training is kind of standard. But this isn't the point, what I mean is this is an "easy" task for the model to solve.  
  
  ---
  
  