Answers.Body: <p>The random forest (rf) is the perfectly parallel ensemble of <a href="https://www.geeksforgeeks.org/cart-classification-and-regression-tree-in-machine-learning/" rel="nofollow noreferrer">CART</a> learners. It uses <a href="https://www.analyticsvidhya.com/blog/2021/03/how-to-select-best-split-in-decision-trees-gini-impurity/" rel="nofollow noreferrer">Gini impurity</a> to inform its split locations, and ensemble summary (mean, mode) to track error.</p>
  <p>The gradient boosted machine (gbm) is the perfectly serial ensemble of CART learners. It can use the Adaboost method of weighting error, though gradient boosting tends to be preferred.  The first tree is the predictor, and all others try and resolve the error.</p>
  <p>There is a multiplicity of ensembles that are not 100% parallel or 0% parallel.</p>
  <p>I have had decent results by predicting using the RF, computing the Error, and then using a gbm to try and clean up the error.  It is something that has been &quot;on the menu&quot; for ~20 years since those learners came out.</p>
  <p><strong>EDIT</strong>:<br />
  These folks talk about it some.  I don't know if they made a random-forest of gradient boosted trees, or a gradient boosted tree of random-forests (its been a while since I went through it).</p>
  <p><a href="https://www.jmlr.org/papers/volume10/tuv09a/tuv09a.pdf" rel="nofollow noreferrer">https://www.jmlr.org/papers/volume10/tuv09a/tuv09a.pdf</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-28T21:34:59.270Z
Answers.LastActivityDate: 2023-05-30T14:51:05.717Z
Answers.LastEditDate: 2023-05-30T14:51:05.717Z
Answers.UserDisplayName: EngrStudent
Body: <p>I know the random forest is a bagging technique. But what if my random forest overfits on a dataset, so I reduce the depth of the decision tree and now it is underfitting. In this scenario, can I take the under-fitted random forest with little depth and try to boost it?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-07-17T13:06:25.463Z
Comments.Text: Random forest does not overfit. Check this - https://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm#remarks
Comments.UserDisplayName: naive
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-12-28T21:09:11.533Z
Comments.Text: @naive - any learner can overfit, though the RF tries hard not to do so.
Comments.UserDisplayName: EngrStudent
Tags: <machine-learning><decision-trees><random-forests>
Title: Can I apply AdaBoost on a random forest?
UserDisplayName: Swakshar Deb
ThreadText: # Can I apply AdaBoost on a random forest?
  
  (Asked by: username_0 on 2020-07-17)
  
  I know the random forest is a bagging technique. But what if my random forest overfits on a dataset, so I reduce the depth of the decision tree and now it is underfitting. In this scenario, can I take the under-fitted random forest with little depth and try to boost it?
  
  **Tags:** machine-learning, decision-trees, random-forests
  > **Comments:**
  > **username_1**: Random forest does not overfit. Check this - https://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm#remarks  
  > **username_2**: @naive - any learner can overfit, though the RF tries hard not to do so.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-12-28. Score: 1
  
  The random forest (rf) is the perfectly parallel ensemble of [CART](https://www.geeksforgeeks.org/cart-classification-and-regression-tree-in-machine-learning/) learners. It uses [Gini impurity](https://www.analyticsvidhya.com/blog/2021/03/how-to-select-best-split-in-decision-trees-gini-impurity/) to inform its split locations, and ensemble summary (mean, mode) to track error.
  
  The gradient boosted machine (gbm) is the perfectly serial ensemble of CART learners. It can use the Adaboost method of weighting error, though gradient boosting tends to be preferred. The first tree is the predictor, and all others try and resolve the error.
  
  There is a multiplicity of ensembles that are not 100% parallel or 0% parallel.
  
  I have had decent results by predicting using the RF, computing the Error, and then using a gbm to try and clean up the error. It is something that has been "on the menu" for ~20 years since those learners came out.
  
  **EDIT**:  
  These folks talk about it some. I don't know if they made a random-forest of gradient boosted trees, or a gradient boosted tree of random-forests (its been a while since I went through it).
  
  <https://www.jmlr.org/papers/volume10/tuv09a/tuv09a.pdf>
  
  
  ---
  
  