<p>random forest's feature importances are not reliable and you should probably avoid them. Instead you can use <code>permutation_importance</code>: <a href="https://scikit-learn.org/stable/auto_examples/inspection/plot_permutation_importance.html#sphx-glr-auto-examples-inspection-plot-permutation-importance-py" rel="nofollow noreferrer">https://scikit-learn.org/stable/auto_examples/inspection/plot_permutation_importance.html#sphx-glr-auto-examples-inspection-plot-permutation-importance-py</a></p>

CC BY-SA 4.0
2020-01-16T20:31:37.743Z
"random forest's feature importances are not reliable and you should probably avoid them" is a very strong statement which does not hold. While random forest variable importance estimates are known to be biased under certain conditions (e.g. gini impurity-based estimates on a mix of continuous and categorical variables), they can provide relatively good estimates in other situations.
Jonathan
CC BY-SA 4.0
2020-01-16T22:18:50.143Z
@adrin thanks. I want to get the features values in "permutation_importance". 
As in "importances = clf.feature_importances_", to build a graph . 
If I request"result.importances", I get a lot of items on each index.
It turns out to get only indexes. Which function should I turn to?
user287629
CC BY-SA 4.0
2020-01-16T17:41:15.850Z
2020-01-16T17:41:15.850Z
adrin
<p>The description of feature selection based on a random forest uses trees without pruning.
Do I need to use tree pruning?
The thing is, if I don't cut the trees, the forest will retrain.</p>

<p>Below in the picture is the importance of features based on 500 trees without pruning.
<a href="https://i.sstatic.net/Az5sw.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Az5sw.jpg" alt="enter image description here"></a></p>

<p>With a depth of 3.
<a href="https://i.sstatic.net/nKt29.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/nKt29.jpg" alt="enter image description here"></a></p>

<p>I always use the last four signs 27, 28, 29, 30.
And I try to add to them signs from 0 to 26 by means of cycles, going through possible combinations.
Empirically, I assume that the trait number 0, 26 is significant.
But, on both pictures it is not visible. Although the quality of classification with the addition of 0, 26 has improved.</p>

<machine-learning><classification><python><scikit-learn>
Interpretation of feature selection based on the model
user287629
# Interpretation of feature selection based on the model

(Asked by: username_0 on 2020-01-02)

The description of feature selection based on a random forest uses trees without pruning.
Do I need to use tree pruning?
The thing is, if I don't cut the trees, the forest will retrain.

Below in the picture is the importance of features based on 500 trees without pruning.
[![enter image description here](https://i.sstatic.net/Az5sw.jpg)](https://i.sstatic.net/Az5sw.jpg)

With a depth of 3.
[![enter image description here](https://i.sstatic.net/nKt29.jpg)](https://i.sstatic.net/nKt29.jpg)

I always use the last four signs 27, 28, 29, 30.
And I try to add to them signs from 0 to 26 by means of cycles, going through possible combinations.
Empirically, I assume that the trait number 0, 26 is significant.
But, on both pictures it is not visible. Although the quality of classification with the addition of 0, 26 has improved.

**Tags:** machine-learning, classification, python, scikit-learn


---

## 1 answers

### Answer by username_1 on 2020-01-16. Score: 0 | Accepted answer

random forest's feature importances are not reliable and you should probably avoid them. Instead you can use `permutation_importance`: <https://scikit-learn.org/stable/auto_examples/inspection/plot_permutation_importance.html#sphx-glr-auto-examples-inspection-plot-permutation-importance-py>

> **Comments:**
> **username_2**: "random forest's feature importances are not reliable and you should probably avoid them" is a very strong statement which does not hold. While random forest variable importance estimates are known to be biased under certain conditions (e.g. gini impurity-based estimates on a mix of continuous and categorical variables), they can provide relatively good estimates in other situations.  
> **username_0**: @username_1 thanks. I want to get the features values in "permutation_importance".  As in "importances = clf.feature_importances_", to build a graph .  If I request"result.importances", I get a lot of items on each index. It turns out to get only indexes. Which function should I turn to?  

---

