What is the main purpose of Feature Selection? I have a small medical dataset (200 samples) that contains only 6 cases of the condition I am trying to predict using machine learning. So far, the dataset is not proving useful for predicting the target variable and is resulting in models with 0% recall and precision, probably due to the scarcity of the minority class.
However, in order to learn from the dataset, I applied Feature Selection techniques to deduct what features are useful in predicting the target variable and see if this supports or contradicts previous literature on the matter.
When I reran my models using the reduced dataset, this still resulted in 0% recall and precision. So the prediction performance has not improved using feature selection. But the features returned by the applying Feature Selection have given me more insight into the data.
So my question is, is the purpose of Feature Selection:

*

*to improve prediction performance

*or can the purpose be identifying relevant features in the prediction and learning more about the dataset

So in other words, is Feature Selection just a tool to achieve improved performance, or can it be an end in itself?


What is the main purpose of Feature Selection?

Feature selection is probably the most important topic in model building. Indeed asking clarification about the purpose of variables selection looks like asking clarification about the purpose of the model itself.
In my view most models, for example most regression models, can have two main goals: Causal inference and prediction. See here: What is the relationship between minimizing prediction error versus parameter estimation error?
and links therein.
Therefore

So my question is, is the purpose of Feature Selection:

*

*to improve prediction performance


this is surely a right purpose for features selection. In particular it is the purpose if the model is built for prediction.

So my question is, is the purpose of Feature Selection:

*

*or can the purpose be identifying relevant features in the prediction and learning more about the dataset


this sentences is a bit tautological "the purpose of features selection is ... the purpose of identifying relevant features".
Moreover your point sound like a conflation between prediction and something like explanation. This is not a good idea, read here: Minimizing bias in explanatory modeling, why? (Galit Shmueli's "To Explain or to Predict")
The second scope for variables selection is related with causal inference. In short this purpose is to avoid endogeneity and related distortions in estimation of causal parameters of interest. In other words, variables selection is useful for identification.
See here for example: Which OLS assumptions are colliders violating?
