Why are my elastic net and lasso r-squared measures negative? I'm using sklearn.linear_model.Lasso and sklearn.linear_model.ElasticNet on a model that includes a constant.
I don't expect a model with a constant to perform worse than the average of the data, ie an r-squared of 0.
I have p > n, but I have regressed on a few subset models and it's trivial to find an OLS subset with r-squared > .5.
Below are some of the model metrics. I'm baffled that the r-squared is negative at all, and further baffled that r-squared is below -1.
Lasso(alpha=0.1)
r^2 on test data : -1.609104
ElasticNet(alpha=0.1, l1_ratio=0.7)
r^2 on test data : -1.745874

Related code is open-source here.
I tried a variety of other alphas but did not achieve a non-negative r-squared. I did achieve an r-squared close to -0.003 with an alpha of 10.
Could anyone provide insight into how such metrics are observed?

Update, 10/20/21 - comparative sample scatter below
plt.scatter(y_train, y_train_pred_lasso, color='red')
plt.scatter(y_test, y_test_pred_lasso, color='blue')
plt.show()



PRESS is an out-of-sample measure. Even when you have an intercept, out-of-sample $R^2<0$ is possible. It means that you would have been better off predicting the mean of $y$ every time, regardless of your predictors/features. In other words, you have a poor model that is outperformed by a naïve model.
You say that this is unexpected performance. You're doing machine learning, so prepare yourself to be surprised and disappointed by model performance.
