Answers.Body: <p>Feature scaling happens to be a problem when a model is characterized by having a distance metric (or another kind of numerical evaluation for that matter). Therefore models such as support vector machines, neural networks, distance based clustering methods (e.g. k means) and linear/logistic regression are prone to changes by feature scaling.</p>
  <p>Those which are based on <strong>probability</strong> rather than distances are not scale variant. These include Naive Bayes Classifiers, or decision trees.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-11T02:21:51.450Z
Answers.LastActivityDate: 2021-02-14T15:52:47.123Z
Answers.LastEditDate: 2021-02-14T15:52:47.123Z
Answers.UserDisplayName: Ghostpunk
Answers.Body: <p>Scaling only makes sense when there is something that reacts to that scale. Decision Trees though, just make a cut at a certain number.</p>
  <p>Imagine: For a feature that goes from 0 to 100 a cut at 50 may be improving performance. Scaling this down to 0 to 1 making the cut a 0.5 doesn't change a thing.</p>
  <p>Now on the other hand NN have some kind of activation function (leaving RELu aside) that react differently to input that is above 1. Here Normalization, putting every feature between 0 and 1 makes sense.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-03T07:39:16.543Z
Answers.LastActivityDate: 2020-07-03T07:39:16.543Z
Answers.UserDisplayName: N. Kiefer
Body: <p>Feature scaling, in general, is an important stage in the data preprocessing pipeline.</p>
  <p>Decision Tree and Random Forest algorithms, though, are scale-invariant - i.e. they work fine without feature scaling. Why is that?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-02-15T23:08:02.033Z
Comments.Text: what problem are you trying to solve?  this is a very generalized question.  Are you trying to understand the limitations of the Random Forest Classifier and Regressor
Comments.UserDisplayName: Golden Lion
Tags: <decision-trees><random-forests><feature-engineering>
Title: Why are decision trees and random forests scale invariant?
UserDisplayName: stoic-santiago
ThreadText: # Why are decision trees and random forests scale invariant?
  
  (Asked by: username_0 on 2020-07-02)
  
  Feature scaling, in general, is an important stage in the data preprocessing pipeline.
  
  Decision Tree and Random Forest algorithms, though, are scale-invariant - i.e. they work fine without feature scaling. Why is that?
  
  **Tags:** decision-trees, random-forests, feature-engineering
  
  > **Comments:**
  > **username_1**: what problem are you trying to solve?  this is a very generalized question.  Are you trying to understand the limitations of the Random Forest Classifier and Regressor  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2020-07-03. Score: 4 | Accepted answer
  
  Scaling only makes sense when there is something that reacts to that scale. Decision Trees though, just make a cut at a certain number.
  
  Imagine: For a feature that goes from 0 to 100 a cut at 50 may be improving performance. Scaling this down to 0 to 1 making the cut a 0.5 doesn't change a thing.
  
  Now on the other hand NN have some kind of activation function (leaving RELu aside) that react differently to input that is above 1. Here Normalization, putting every feature between 0 and 1 makes sense.
  
  
  ---
  
  ### Answer by username_2 on 2021-02-11. Score: 2
  
  Feature scaling happens to be a problem when a model is characterized by having a distance metric (or another kind of numerical evaluation for that matter). Therefore models such as support vector machines, neural networks, distance based clustering methods (e.g. k means) and linear/logistic regression are prone to changes by feature scaling.
  
  Those which are based on **probability** rather than distances are not scale variant. These include Naive Bayes Classifiers, or decision trees.
  
  
  ---
  
  