How to approach regression problem with ~30 datapoints and thousands of features? I'm trying to build a model that predicts the 6 numerical target values from a molecule. I encode a molecule with "descriptors", which give me about 2000 numerical features that describe the molecule in various ways. I have no chemical insight in this problem, so I have no idea which features may be relevant for predicting the 6 target values.
My dataset is very small: I only have the responses (the 6 target values) for 30 molecules, as it is very costly to test a new molecule. My goal is to create a model that can predict these 6 target values for a molecule, so I can choose which molecule to try next to see if it has the performance I wish.
I've read a lot about QSPR and QSAR (quantitative structure property/activity relationship, in essence predicting something about a molecule), but I'm overwhelmed by the fact that everyone uses a different machine learning model for their problem. Also, in my case I have much fewer datapoints than the typical QSPR models.
I'm equally overwhelmed by the variety of feature selection and dimensionality reduction algorithms: PCA, univariate feature selection, feature elimination... Can I know that some algorithms have more chances of being useful for my problem beforehand, or do I just have to test them all and see for myself?
To sum up, my questions are:

*

*What machine learning model would be indicated for this?

*What feature selection models would be useful, if any?

*Should I build an independent model for each of my 6 target responses, or try a multiple regression model? The responses are correlated.

My guesses would be perhaps SVM and random forest, as they have some feature selection inherent to the model and robustness to overfitting. But I'm inexperienced in the field of ML and any input would be greatly appreciated!

This situation is common for example in Neuro imaging or genetics. The common recommendation there is to use linear regularized models such as ridge regression, elastic net, linear SVM or linear gaussian process. Often it is NOT recommended to tune parameters or try many models, because with such a small sample size, The model selection bias due to vibration effects is more dangerous than having sub optimal lambda. Also all these linear models perform very similar anyway.
The core problem is not that you don't have enough data to fit your models, but that you don't have enough data to estimate if they are any good or which approach is better.
PCA is closely related to ridge regression and L2 penalization, so running PCA regression or those regularized models also won't produce much of a difference. And again, you don't have enough data to detect model improvement that is not large.
