<p>Generally if a model A performs worse than model B on &quot;Test set&quot; while performing better on &quot;Train/Val set&quot; then model A has suffered from overfitting.</p>
<p><a href="https://i.sstatic.net/O79aM.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/O79aM.png" alt="What is overfitting?" /></a></p>
<p>So, in your case Model 1 should give you long term reliability against unseen data.</p>

CC BY-SA 4.0
2022-08-08T12:46:45.507Z
Thank you very much for your answer Marib.
Rowan Barua
CC BY-SA 4.0
2022-08-08T13:07:57.610Z
If you are satisfied with the answer, can you mark it as accepted
Marib Sultan
CC BY-SA 4.0
2022-08-08T11:50:36.270Z
2022-08-08T11:50:36.270Z
Marib Sultan
<p>I am looking for some advice regarding the best choice of binary classification model based on training, validation and test set results. Model 1 (results in 1st image) shows better test set results than Model 2, but Model 2 (results in 2nd image) shows results that seem more intuitive to me with better training set performance than its test set performance. I feel as if the Model 1 test set results might have been a bit of a fluke, whereas Model 2 appears more like a well trained model with more long-term reliability.</p>
<p>Any advice on this is much appreciated.</p>
<p><a href="https://i.sstatic.net/BEDmQ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/BEDmQ.png" alt="Model 1 Results" /></a></p>
<p><a href="https://i.sstatic.net/Hwpxe.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Hwpxe.png" alt="Model 2 Results" /></a></p>

<binary-classification><confusion-matrix>
Binary Classification Model Comparison - Interpretation of Training, Test and Validation Set Performance
Rowan Barua
# Binary Classification Model Comparison - Interpretation of Training, Test and Validation Set Performance

(Asked by: username_0 on 2022-08-08)

I am looking for some advice regarding the best choice of binary classification model based on training, validation and test set results. Model 1 (results in 1st image) shows better test set results than Model 2, but Model 2 (results in 2nd image) shows results that seem more intuitive to me with better training set performance than its test set performance. I feel as if the Model 1 test set results might have been a bit of a fluke, whereas Model 2 appears more like a well trained model with more long-term reliability.

Any advice on this is much appreciated.

[![Model 1 Results](https://i.sstatic.net/BEDmQ.png)](https://i.sstatic.net/BEDmQ.png)

[![Model 2 Results](https://i.sstatic.net/Hwpxe.png)](https://i.sstatic.net/Hwpxe.png)

**Tags:** binary-classification, confusion-matrix


---

## 1 answers

### Answer by username_1 on 2022-08-08. Score: 0 | Accepted answer

Generally if a model A performs worse than model B on "Test set" while performing better on "Train/Val set" then model A has suffered from overfitting.

[![What is overfitting?](https://i.sstatic.net/O79aM.png)](https://i.sstatic.net/O79aM.png)

So, in your case Model 1 should give you long term reliability against unseen data.

> **Comments:**
> **username_0**: Thank you very much for your answer Marib.  
> **username_1**: If you are satisfied with the answer, can you mark it as accepted  

---

