Answers.Body: <p>Decision Tree learners, on their own, are not a good way to deal with imbalanced data. The most commonly used algorithms, by default, make no attempt to address this problem.</p>
  
  <p>If you look carefully at the three sources you post, you will find that they actually all agree on this point.</p>
  
  <p>Two of the sources actually propose methods of <em>addressing</em> this shortcoming, by making adjustments to the decision tree learning algorithms. The proposed adjustments are essentially standard solutions to these problems, being applied to decision trees.</p>
  
  <p>An example technique, discussed in the first paper you reference, is changing the weightings of the classes. An inefficient/approximate way to do this is to increase the number of examples from the minority class. For example, if you had an 80/20 split, you could add 3 new copies of each minority class example to move to an 80/80 = 50/50 split. Of course, if you add new data points, your algorithm may take longer to run. Instead, you can just modify the weightings of the classes in your optimization function. This approach is algorithm-specific, and will depend on your loss function, but achieves the same effect, just without needing to increase the number of points you use.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-19T21:56:40.013Z
Answers.LastActivityDate: 2019-11-20T13:59:43.477Z
Answers.LastEditDate: 2019-11-20T13:59:43.477Z
Answers.UserDisplayName: John Doucette
Body: <p>Decision trees and random forests may or <em>not</em> be more suited to solve supervised learning problems with imbalanced labels (or classes) in datasets. For example, see the article <a href="https://statistics.berkeley.edu/sites/default/files/tech-reports/666.pdf" rel="nofollow noreferrer">Using Random Forest to Learn Imbalanced Data</a>, <a href="https://stats.stackexchange.com/q/242833/82135">this Stats SE question</a> and <a href="https://towardsdatascience.com/working-with-highly-imbalanced-datasets-in-machine-learning-projects-c70c5f2a7b16" rel="nofollow noreferrer">this Medium post</a>. The information across these sources does not seem to be consistent.</p>
  
  <p>How could decision tree learning algorithms cope with imbalanced classes?</p>
  
Tags: <machine-learning><datasets><supervised-learning><decision-trees><random-forests>
Title: How could decision tree learning algorithms cope with imbalanced classes?
UserDisplayName: jennifer ruurs
ThreadText: # How could decision tree learning algorithms cope with imbalanced classes?
  
  (Asked by: username_0 on 2019-11-18)
  
  Decision trees and random forests may or *not* be more suited to solve supervised learning problems with imbalanced labels (or classes) in datasets. For example, see the article [Using Random Forest to Learn Imbalanced Data](https://statistics.berkeley.edu/sites/default/files/tech-reports/666.pdf), [this Stats SE question](https://stats.stackexchange.com/q/242833/82135) and [this Medium post](https://towardsdatascience.com/working-with-highly-imbalanced-datasets-in-machine-learning-projects-c70c5f2a7b16). The information across these sources does not seem to be consistent.
  
  How could decision tree learning algorithms cope with imbalanced classes?
  
  **Tags:** machine-learning, datasets, supervised-learning, decision-trees, random-forests
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-11-19. Score: 4
  
  Decision Tree learners, on their own, are not a good way to deal with imbalanced data. The most commonly used algorithms, by default, make no attempt to address this problem.
  
  If you look carefully at the three sources you post, you will find that they actually all agree on this point.
  
  Two of the sources actually propose methods of *addressing* this shortcoming, by making adjustments to the decision tree learning algorithms. The proposed adjustments are essentially standard solutions to these problems, being applied to decision trees.
  
  An example technique, discussed in the first paper you reference, is changing the weightings of the classes. An inefficient/approximate way to do this is to increase the number of examples from the minority class. For example, if you had an 80/20 split, you could add 3 new copies of each minority class example to move to an 80/80 = 50/50 split. Of course, if you add new data points, your algorithm may take longer to run. Instead, you can just modify the weightings of the classes in your optimization function. This approach is algorithm-specific, and will depend on your loss function, but achieves the same effect, just without needing to increase the number of points you use.
  
  
  ---
  
  