Does the reliability of feature importance depend on model performance? I am wondering if there is a relationship between the performance quality of a predictive model (e.g., a classifier) and the reliability of corresponding feature importance values?
To give an example: I created two binary classifiers using two different sets of features. One classifier performs slightly better than the other. However, both perform not that great (balanced accuracies are around 65% and 55%). I calculated feature importance scores using permutation importance. Is it valid to say that both lists of importance values are equally reliable/interpretable?
Any help or literature recommendations are highly appreciated!

Yes feature importance is computed based on the impact on performance of the model. A feature with high importance means it has large impact on the performance for example, from 90% it came down to 70%. And when the performance measures are itself on the lower side, impact on performance does not make much sense. In a way you could relate performance and feature importance.
