Answers.Body: <p>random forest's feature importances are not reliable and you should probably avoid them. Instead you can use <code>permutation_importance</code>: <a href="https://scikit-learn.org/stable/auto_examples/inspection/plot_permutation_importance.html#sphx-glr-auto-examples-inspection-plot-permutation-importance-py" rel="nofollow noreferrer">https://scikit-learn.org/stable/auto_examples/inspection/plot_permutation_importance.html#sphx-glr-auto-examples-inspection-plot-permutation-importance-py</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-16T20:31:37.743Z
Answers.Text: "random forest's feature importances are not reliable and you should probably avoid them" is a very strong statement which does not hold. While random forest variable importance estimates are known to be biased under certain conditions (e.g. gini impurity-based estimates on a mix of continuous and categorical variables), they can provide relatively good estimates in other situations.
Answers.UserDisplayName: Jonathan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-16T22:18:50.143Z
Answers.Text: @adrin thanks. I want to get the features values in "permutation_importance". 
  As in "importances = clf.feature_importances_", to build a graph . 
  If I request"result.importances", I get a lot of items on each index.
  It turns out to get only indexes. Which function should I turn to?
Answers.UserDisplayName: user287629
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-16T17:41:15.850Z
Answers.LastActivityDate: 2020-01-16T17:41:15.850Z
Answers.UserDisplayName: adrin
Body: <p>The description of feature selection based on a random forest uses trees without pruning.
  Do I need to use tree pruning?
  The thing is, if I don't cut the trees, the forest will retrain.</p>
  
  <p>Below in the picture is the importance of features based on 500 trees without pruning.
  <a href="https://i.sstatic.net/Az5sw.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Az5sw.jpg" alt="enter image description here"></a></p>
  
  <p>With a depth of 3.
  <a href="https://i.sstatic.net/nKt29.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/nKt29.jpg" alt="enter image description here"></a></p>
  
  <p>I always use the last four signs 27, 28, 29, 30.
  And I try to add to them signs from 0 to 26 by means of cycles, going through possible combinations.
  Empirically, I assume that the trait number 0, 26 is significant.
  But, on both pictures it is not visible. Although the quality of classification with the addition of 0, 26 has improved.</p>
  
Tags: <machine-learning><classification><python><scikit-learn>
Title: Interpretation of feature selection based on the model
UserDisplayName: user287629
ThreadText: # Interpretation of feature selection based on the model
  
  (Asked by: username_0 on 2020-01-02)
  
  The description of feature selection based on a random forest uses trees without pruning.
  Do I need to use tree pruning?
  The thing is, if I don't cut the trees, the forest will retrain.
  
  Below in the picture is the importance of features based on 500 trees without pruning.
  [![enter image description here](https://i.sstatic.net/Az5sw.jpg)](https://i.sstatic.net/Az5sw.jpg)
  
  With a depth of 3.
  [![enter image description here](https://i.sstatic.net/nKt29.jpg)](https://i.sstatic.net/nKt29.jpg)
  
  I always use the last four signs 27, 28, 29, 30.
  And I try to add to them signs from 0 to 26 by means of cycles, going through possible combinations.
  Empirically, I assume that the trait number 0, 26 is significant.
  But, on both pictures it is not visible. Although the quality of classification with the addition of 0, 26 has improved.
  
  **Tags:** machine-learning, classification, python, scikit-learn
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-01-16. Score: 0 | Accepted answer
  
  random forest's feature importances are not reliable and you should probably avoid them. Instead you can use `permutation_importance`: <https://scikit-learn.org/stable/auto_examples/inspection/plot_permutation_importance.html#sphx-glr-auto-examples-inspection-plot-permutation-importance-py>
  
  > **Comments:**
  > **username_2**: "random forest's feature importances are not reliable and you should probably avoid them" is a very strong statement which does not hold. While random forest variable importance estimates are known to be biased under certain conditions (e.g. gini impurity-based estimates on a mix of continuous and categorical variables), they can provide relatively good estimates in other situations.  
  > **username_0**: @username_1 thanks. I want to get the features values in "permutation_importance".  As in "importances = clf.feature_importances_", to build a graph .  If I request"result.importances", I get a lot of items on each index. It turns out to get only indexes. Which function should I turn to?  
  
  ---
  
  