Question about linear machine learning models, positive coefficients, correlated features, and overfitting I am experimenting with a few different linear models from SKLearn. I am using a dataset with about 600 features and 350,000 samples. I have noticed that I get extreme overfitting unless I force the coefficients to be positive. Once I force coefficients to be positive I get a good match between in-sample and out of sample performance.
There is high correlation between the features, with many pairs above 0.7 measured by spearman correlation.
So my questions is whether the overfitting is due to the correlated features? Somehow when the algorithm is able to subtract two highly correlated features its able to overfit, but when it can only add them together it can't?


So my questions is whether the overfitting is due to the correlated features?

Doubtful.  Correlated features affect the stability of the coefficients, not the predictions.
What kinds of models are you using?  You have nigh 3 orders of magnitude more observations than variables, and so there should not be much over fitting from a linear model unless you're taking tons of pairwise interactions.
