<p>Feature scaling happens to be a problem when a model is characterized by having a distance metric (or another kind of numerical evaluation for that matter). Therefore models such as support vector machines, neural networks, distance based clustering methods (e.g. k means) and linear/logistic regression are prone to changes by feature scaling.</p>
<p>Those which are based on <strong>probability</strong> rather than distances are not scale variant. These include Naive Bayes Classifiers, or decision trees.</p>

CC BY-SA 4.0
2021-02-11T02:21:51.450Z
2021-02-14T15:52:47.123Z
2021-02-14T15:52:47.123Z
Ghostpunk
<p>Scaling only makes sense when there is something that reacts to that scale. Decision Trees though, just make a cut at a certain number.</p>
<p>Imagine: For a feature that goes from 0 to 100 a cut at 50 may be improving performance. Scaling this down to 0 to 1 making the cut a 0.5 doesn't change a thing.</p>
<p>Now on the other hand NN have some kind of activation function (leaving RELu aside) that react differently to input that is above 1. Here Normalization, putting every feature between 0 and 1 makes sense.</p>

CC BY-SA 4.0
2020-07-03T07:39:16.543Z
2020-07-03T07:39:16.543Z
N. Kiefer
<p>Feature scaling, in general, is an important stage in the data preprocessing pipeline.</p>
<p>Decision Tree and Random Forest algorithms, though, are scale-invariant - i.e. they work fine without feature scaling. Why is that?</p>

CC BY-SA 4.0
2021-02-15T23:08:02.033Z
what problem are you trying to solve?  this is a very generalized question.  Are you trying to understand the limitations of the Random Forest Classifier and Regressor
Golden Lion
<decision-trees><random-forests><feature-engineering>
Why are decision trees and random forests scale invariant?
stoic-santiago
# Why are decision trees and random forests scale invariant?

(Asked by: username_0 on 2020-07-02)

Feature scaling, in general, is an important stage in the data preprocessing pipeline.

Decision Tree and Random Forest algorithms, though, are scale-invariant - i.e. they work fine without feature scaling. Why is that?

**Tags:** decision-trees, random-forests, feature-engineering

> **Comments:**
> **username_1**: what problem are you trying to solve?  this is a very generalized question.  Are you trying to understand the limitations of the Random Forest Classifier and Regressor  

---

## 2 answers

### Answer by username_3 on 2020-07-03. Score: 4 | Accepted answer

Scaling only makes sense when there is something that reacts to that scale. Decision Trees though, just make a cut at a certain number.

Imagine: For a feature that goes from 0 to 100 a cut at 50 may be improving performance. Scaling this down to 0 to 1 making the cut a 0.5 doesn't change a thing.

Now on the other hand NN have some kind of activation function (leaving RELu aside) that react differently to input that is above 1. Here Normalization, putting every feature between 0 and 1 makes sense.


---

### Answer by username_2 on 2021-02-11. Score: 2

Feature scaling happens to be a problem when a model is characterized by having a distance metric (or another kind of numerical evaluation for that matter). Therefore models such as support vector machines, neural networks, distance based clustering methods (e.g. k means) and linear/logistic regression are prone to changes by feature scaling.

Those which are based on **probability** rather than distances are not scale variant. These include Naive Bayes Classifiers, or decision trees.


---

