Why use regularization instead of feature selection for logistic regression? For a non-linearly separable problem, when there are enough features, we can make the data linearly separable. It seems to me that for logistic regression, the reason of overfitting is always excessive number of features. So why people mostly use l1, specially l2 regularization to shrink $w$ but not use feature selection? With the correct features (that can't perfectly sperate the data), does $w$ also become large?

As you quoted: It seems to me that for logistic regression, the reason of overfitting is always excessive number of features. So why people mostly use l1, specially l2 regularisation to shrink  but not use feature selection?

If you have the problem of Overfitting in your ML model, you tend to
penalize the features, which are making your model overfit. This can
be achieved by the LASSO/RIDGE techniques and are very sustainable
methodologies too.

Ok, now supposedly, you want to handle feature selection scenario, then there are multiple ways to do that also,

*

*Correlation

*PCA

*SelectKBest

*Entropy and much more techniques.

But the challenge would be here to apply each methodology and understand the behaviour/importance of each feature post which use those certain features for ML Modelling. But, when the technique of LASSO and RIDGE performs feature selection as well as Modelling for you in one shot, then why to do so much cumbersome process.
Hope this clarifies.
