Answers.Body: <p>This is a sign of <a href="https://en.wikipedia.org/wiki/Overfitting" rel="nofollow noreferrer">overfitting</a>.</p>
  
  <p>As you make your trees deeper, it becomes possible to "memorize" the data: each leaf of the tree is just a single point. The trees begin to learn patterns that do not exist. When you try out these patterns on new data (which is what cross-validation is imitating), then the patterns do not work, and your model fails to generalize.</p>
  
  <p>The main piece of information to draw from this plot is that the optimum tree depth is about 15.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-19T05:53:57.520Z
Answers.Text: Isn't the opposite true, that I am underfitting? Because the we do first cross-validation and evaluate on the test set how the cross-validation model behaves on the test set. If the scores of the test set is lower than cross-validation than it is overfitting?
Answers.UserDisplayName: jennifer ruurs
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-19T11:53:51.350Z
Answers.Text: And is the second one better than the first one?
Answers.UserDisplayName: jennifer ruurs
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-19T12:45:07.450Z
Answers.Text: @jenniferruurs Lower scores on the test set than the training set can mean overfitting or underfitting. Your plot tells us that depth values below 15 are probably underfitting: we can improve performance by increasing model complexity. Depth values above 15 are probably overfitting: we can improve performance by reducing model complexity. Neither line is "better" than the other. They mean different things.
Answers.UserDisplayName: John Doucette
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-19T12:54:49.070Z
Answers.Text: I mean i added a second plot, and is the second plot a better model than the first one?
Answers.UserDisplayName: jennifer ruurs
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-19T18:48:54.710Z
Answers.Text: In assessing model quality, you should only use test it validation performance.  In both of your graphs, the cross validation curves are essentially the same. The best models occur at a depth of 15, and they have about 84% accuracy in both cases.
Answers.UserDisplayName: John Doucette
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-18T23:18:29.637Z
Answers.LastActivityDate: 2019-10-18T23:18:29.637Z
Answers.UserDisplayName: John Doucette
Body: <p>Bellow I have a validation plot
  How should I interpret this validation plot?
  Is my data underfitting? What else can be seen from this?
  Which one is the best?</p>
  
  <p>What does it mean that the right line is growing and green line decrease (slightly) for example after 15?</p>
  
  <p><a href="https://i.sstatic.net/ONEtk.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ONEtk.jpg" alt="enter image description here"></a></p>
  
  <p>Second random forrest
  <a href="https://i.sstatic.net/yHk3P.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/yHk3P.png" alt="enter image description here"></a></p>
  
Tags: <machine-learning><cross-validation><random-forests>
Title: How should I interpret this validation plot?
UserDisplayName: jennifer ruurs
ThreadText: # How should I interpret this validation plot?
  
  (Asked by: username_0 on 2019-10-18)
  
  Bellow I have a validation plot
  How should I interpret this validation plot?
  Is my data underfitting? What else can be seen from this?
  Which one is the best?
  
  What does it mean that the right line is growing and green line decrease (slightly) for example after 15?
  
  [![enter image description here](https://i.sstatic.net/ONEtk.jpg)](https://i.sstatic.net/ONEtk.jpg)
  
  Second random forrest
  [![enter image description here](https://i.sstatic.net/yHk3P.png)](https://i.sstatic.net/yHk3P.png)
  
  **Tags:** machine-learning, cross-validation, random-forests
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-10-18. Score: 2 | Accepted answer
  
  This is a sign of [overfitting](https://en.wikipedia.org/wiki/Overfitting).
  
  As you make your trees deeper, it becomes possible to "memorize" the data: each leaf of the tree is just a single point. The trees begin to learn patterns that do not exist. When you try out these patterns on new data (which is what cross-validation is imitating), then the patterns do not work, and your model fails to generalize.
  
  The main piece of information to draw from this plot is that the optimum tree depth is about 15.
  
  > **Comments:**
  > **username_0**: Isn't the opposite true, that I am underfitting? Because the we do first cross-validation and evaluate on the test set how the cross-validation model behaves on the test set. If the scores of the test set is lower than cross-validation than it is overfitting?  
  > **username_0**: And is the second one better than the first one?  
  > **username_1**: @username_0 Lower scores on the test set than the training set can mean overfitting or underfitting. Your plot tells us that depth values below 15 are probably underfitting: we can improve performance by increasing model complexity. Depth values above 15 are probably overfitting: we can improve performance by reducing model complexity. Neither line is "better" than the other. They mean different things.  
  > **username_0**: I mean i added a second plot, and is the second plot a better model than the first one?  
  > **username_1**: In assessing model quality, you should only use test it validation performance.  In both of your graphs, the cross validation curves are essentially the same. The best models occur at a depth of 15, and they have about 84% accuracy in both cases.  
  
  ---
  
  