Answers.Body: <p>Generally if a model A performs worse than model B on &quot;Test set&quot; while performing better on &quot;Train/Val set&quot; then model A has suffered from overfitting.</p>
  <p><a href="https://i.sstatic.net/O79aM.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/O79aM.png" alt="What is overfitting?" /></a></p>
  <p>So, in your case Model 1 should give you long term reliability against unseen data.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-08T12:46:45.507Z
Answers.Text: Thank you very much for your answer Marib.
Answers.UserDisplayName: Rowan Barua
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-08T13:07:57.610Z
Answers.Text: If you are satisfied with the answer, can you mark it as accepted
Answers.UserDisplayName: Marib Sultan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-08T11:50:36.270Z
Answers.LastActivityDate: 2022-08-08T11:50:36.270Z
Answers.UserDisplayName: Marib Sultan
Body: <p>I am looking for some advice regarding the best choice of binary classification model based on training, validation and test set results. Model 1 (results in 1st image) shows better test set results than Model 2, but Model 2 (results in 2nd image) shows results that seem more intuitive to me with better training set performance than its test set performance. I feel as if the Model 1 test set results might have been a bit of a fluke, whereas Model 2 appears more like a well trained model with more long-term reliability.</p>
  <p>Any advice on this is much appreciated.</p>
  <p><a href="https://i.sstatic.net/BEDmQ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/BEDmQ.png" alt="Model 1 Results" /></a></p>
  <p><a href="https://i.sstatic.net/Hwpxe.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Hwpxe.png" alt="Model 2 Results" /></a></p>
  
Tags: <binary-classification><confusion-matrix>
Title: Binary Classification Model Comparison - Interpretation of Training, Test and Validation Set Performance
UserDisplayName: Rowan Barua
ThreadText: # Binary Classification Model Comparison - Interpretation of Training, Test and Validation Set Performance
  
  (Asked by: username_0 on 2022-08-08)
  
  I am looking for some advice regarding the best choice of binary classification model based on training, validation and test set results. Model 1 (results in 1st image) shows better test set results than Model 2, but Model 2 (results in 2nd image) shows results that seem more intuitive to me with better training set performance than its test set performance. I feel as if the Model 1 test set results might have been a bit of a fluke, whereas Model 2 appears more like a well trained model with more long-term reliability.
  
  Any advice on this is much appreciated.
  
  [![Model 1 Results](https://i.sstatic.net/BEDmQ.png)](https://i.sstatic.net/BEDmQ.png)
  
  [![Model 2 Results](https://i.sstatic.net/Hwpxe.png)](https://i.sstatic.net/Hwpxe.png)
  
  **Tags:** binary-classification, confusion-matrix
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-08-08. Score: 0 | Accepted answer
  
  Generally if a model A performs worse than model B on "Test set" while performing better on "Train/Val set" then model A has suffered from overfitting.
  
  [![What is overfitting?](https://i.sstatic.net/O79aM.png)](https://i.sstatic.net/O79aM.png)
  
  So, in your case Model 1 should give you long term reliability against unseen data.
  
  > **Comments:**
  > **username_0**: Thank you very much for your answer Marib.  
  > **username_1**: If you are satisfied with the answer, can you mark it as accepted  
  
  ---
  
  