Thousands of features and only 70 samples I am working on a regression problem where I have around 5-10 thousand features and have only 65 samples. I am training my algorithm on 55 samples and testing on 10 samples. I am using both Pearson correlation and feature importance using the random forest to remove irrelevant features. In testing data, the random forest algorithm is predicting the average of the training samples.
I have a few questions:
(1) Which ML algorithm should I use when I have very few samples for training (55 samples in my case)?
(2) The Best way of feature selection when we have much more features than samples?

*

*The chances of any predictive model doing well here are very very slim. You need to either get more data, or use domain knowledge to narrow down the number of predictors you consider.

*That said, Lasso regression is the usual choice in situations like this.

