<p>To evaluate your fit, the results on the validation set, as you say, are essential.</p>
<ul>
<li>A good performance on both the training and the validation set is a sign of a good fit</li>
<li>A poor performance on the training set (and therefore on the validations set) is a sign of <em>underfitting</em></li>
<li>A good performance on the training set, coupled with a poor performance on the validation set is a sign of <em>overfitting</em>.</li>
</ul>
<p>As you can see you cannot evaluate based on the training set alone, whether it's a good fit or an overfit of the training set.</p>
<p>However, you stated in a comment that you also got 98% on the validation set. We can therefore conclude that you do have a good fit and a working model.</p>
<p>The very high resulting score is probably due to the fact that you're working on a very <em>easy task</em>. The size shown in the image seems to be 100 samples per class -- this makes me think this is a 'toy' dataset used just as an exercise. Whether this works in the real world, whatever your objective is, depends on how representative this dataset is.</p>

CC BY-SA 4.0
2022-07-06T18:05:19.317Z
I see what you mean. The reason there's only 100 samples is because I split my dataset to be 80% for training and 20% for testing. Would increasing the testing split make it make it more obvious?

I changed the split to be 50/50 just to see the difference. The new report has around 500 samples. It reported 100 training score and 93 cv score.
Abrahim Mahmud
CC BY-SA 4.0
2022-07-07T07:59:46.363Z
The exact size doesn't matter as long as they are both statistically representative, and 80% training is kind of standard. But this isn't the point, what I mean is this is an "easy" task for the model to solve.
Martino
CC BY-SA 4.0
2022-07-06T10:58:17.403Z
2022-07-06T10:58:17.403Z
Martino
<p>I was playing around with some models and training them on a heart disease dataset and found my Extra Tree Classifier reported a 100% accuracy. I was shocked at first since I had never gotten a perfect score report from a model, but I realized that the model is most likely over/underfit. After doing some research, I found that you can determine if a model is over/underfit by comparing the Training Score and Cross Validation Score reports, but after looking at the report, I could not tell if my model was over/underfit or if it really is perfect. (I am new to machine learning and I have high doubts that it is actually perfect.)
Here is the model report:</p>
<p><a href="https://i.sstatic.net/bTJMU.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/bTJMU.png" alt="Extra Tree Classifier Model Report" /></a></p>
<p>Can someone help me figure out if this model is over/underfit? (If I need to give any more data/info to figure this out, please let me know!)</p>

CC BY-SA 4.0
2022-07-06T08:16:02.530Z
"you can determine if a model is over/underfit by comparing the Training Score and Cross Validation Score reports" > did you do that? How's the performance on the validation set?
Martino
CC BY-SA 4.0
2022-07-06T09:17:02.860Z
Based off the report, the training score was 100 and the cross validation score was about 98.
Abrahim Mahmud
CC BY-SA 4.0
2022-07-08T03:35:20.463Z
Turns out the model contained a whole lot of bias because 72% of the dataset was duplicated data. The reason for the CV and training score being so close is because the testing data was basically the same as the training data.
Abrahim Mahmud
<machine-learning><classification><python>
Is My Extra Tree Classifier Model Over/Underfit?
Abrahim Mahmud
# Is My Extra Tree Classifier Model Over/Underfit?

(Asked by: username_0 on 2022-07-05)

I was playing around with some models and training them on a heart disease dataset and found my Extra Tree Classifier reported a 100% accuracy. I was shocked at first since I had never gotten a perfect score report from a model, but I realized that the model is most likely over/underfit. After doing some research, I found that you can determine if a model is over/underfit by comparing the Training Score and Cross Validation Score reports, but after looking at the report, I could not tell if my model was over/underfit or if it really is perfect. (I am new to machine learning and I have high doubts that it is actually perfect.)
Here is the model report:

[![Extra Tree Classifier Model Report](https://i.sstatic.net/bTJMU.png)](https://i.sstatic.net/bTJMU.png)

Can someone help me figure out if this model is over/underfit? (If I need to give any more data/info to figure this out, please let me know!)

**Tags:** machine-learning, classification, python

> **Comments:**
> **username_1**: "you can determine if a model is over/underfit by comparing the Training Score and Cross Validation Score reports" > did you do that? How's the performance on the validation set?  
> **username_0**: Based off the report, the training score was 100 and the cross validation score was about 98.  
> **username_0**: Turns out the model contained a whole lot of bias because 72% of the dataset was duplicated data. The reason for the CV and training score being so close is because the testing data was basically the same as the training data.  

---

## 1 answers

### Answer by username_1 on 2022-07-06. Score: 0 | Accepted answer

To evaluate your fit, the results on the validation set, as you say, are essential.

* A good performance on both the training and the validation set is a sign of a good fit
* A poor performance on the training set (and therefore on the validations set) is a sign of *underfitting*
* A good performance on the training set, coupled with a poor performance on the validation set is a sign of *overfitting*.

As you can see you cannot evaluate based on the training set alone, whether it's a good fit or an overfit of the training set.

However, you stated in a comment that you also got 98% on the validation set. We can therefore conclude that you do have a good fit and a working model.

The very high resulting score is probably due to the fact that you're working on a very *easy task*. The size shown in the image seems to be 100 samples per class -- this makes me think this is a 'toy' dataset used just as an exercise. Whether this works in the real world, whatever your objective is, depends on how representative this dataset is.

> **Comments:**
> **username_0**: I see what you mean. The reason there's only 100 samples is because I split my dataset to be 80% for training and 20% for testing. Would increasing the testing split make it make it more obvious?  I changed the split to be 50/50 just to see the difference. The new report has around 500 samples. It reported 100 training score and 93 cv score.  
> **username_1**: The exact size doesn't matter as long as they are both statistically representative, and 80% training is kind of standard. But this isn't the point, what I mean is this is an "easy" task for the model to solve.  

---

