Why I am getting bad result for the hypertuning parameters I am trying to learn machine learning with 3 very common datasets (Boston Housing Price, Red Wine Quality, and Bike). I am using 3 ML models (LinReg, Lasso, and Ridge). The LinReg is the default of the Scikit-learn but for Lasso and Ridge, I am doing the hyper tuning.
I have 3 accuracy metrics (MAE, MSE, R2). The overall accuracy is given below
Dataset     Model       MAE         MSE         R2
House       LinReg      2.96        19.60       0.74
House       Lasso       4.58        47.44       0.39
House       Ridge       5.39        65.25       0.16

Dataset     Model       MAE         MSE         R2
Wine        LinReg      0.50        0.42        0.35
Wine        Lasso       0.68        0.65        -0.0008   
Wine        Ridge       0.66        0.62        0.05

Dataset     Model       MAE         MSE             R2
Bike        LinReg      106.56      20812.63        0.37
Bike        Lasso       118.21      25732.92        0.23
Bike        Ridge       136.31      31186.12        0.07    

You can see, the MAE and MSE are always high for the Lasso and Ridge than the LinReg.  Even, I am getting negative R2 for the Wine dataset and Lasso model.
I did very basic data pre-processing and mostly not any feature engineering. As they are very basic datasets.
I am using Optuna for hypertuning. At the time of hypertuning, I am doing 5 fold cross-validation, neg_mean_squared_error as scoring function and set direction="minimize".
For the Lasso and Ridge T am tuning only alpha and the ranges in between 1e-7, 1e2.
def lasso_objective(X_train, y_train, kfolds) -> Callable[[Trial], float]:
    def objective(trial: Trial) -> float:
        args: Dict = dict(
            alpha=trial.suggest_float("alpha", 1e-7, 1e2), 
        )
        estimator = Lasso(**args)
        scores = cross_validate(
            estimator, X=X_train, y=y_train, scoring="neg_mean_squared_error", cv=cv_method, n_jobs=-1
        )
        return float(np.mean(scores["test_score"]))

    return objective


def tune(objective):
    study = optuna.create_study(direction="minimize")
    study.optimize(objective, n_trials=200)
    params = study.best_params
    best_score = study.best_value
    return params

I am supposed to get better results for the Lasso and Ridge (as I am doing tuning here). Could you tell me why I am getting bad MAE and MSE for them? What should I do to get better results for them?

This was an interesting example. Here is some R code for some naive analysis on Boston Housing data (predicting the median value MEDV):
library(mlbench)
library(glmnet)
data("BostonHousing")

n = nrow(BostonHousing)
train_idx = sample(n, round(0.7*n))
train_data = BostonHousing[train_idx, ]
test_data = BostonHousing[-train_idx, ]
lmod = lm(medv ~ ., data = train_data)
mean(lmod$residuals^2) # training MSE for the linear model

r_col = ncol(train_data) # which column contains the response -- here it is the last column

lasso_mod = cv.glmnet(x = data.matrix(train_data[, -r_col]), y = train_data$medv, nfolds = 5)
lasso_mod
ridge_mod = cv.glmnet(x = data.matrix(train_data[, -r_col]), y = train_data$medv, nfolds = 5, alpha = 0)
ridge_mod

loss = function(x,xh) mean((x-xh)^2)

loss(predict(lmod, test_data[,-r_col]), test_data$medv)
loss(predict(lasso_mod, data.matrix(test_data[,-r_col])), test_data$medv)
loss(predict(ridge_mod, data.matrix(test_data[,-r_col])), test_data$medv)


plot(lasso_mod)
plot(ridge_mod)

The numbers for the test MSE of the linear model, the Lasso and the Ridge are 26.7, 33 and 32.2. The latter two are the default models selected by the glment. These numbers are quite variable since this is based on a single training/test split, however, the order seems to be roughly the same. At least the linear model always does better (in my limited set of runs).
Here are the regularization curves produced by the glmnet package:
For the Lasso:

For the Ridge:

They do suggest that the best regularization is essentially no regularization ($\lambda = 0$) which is consistent with the results.
Do you always have to get a better result with Lasso and/or ridge? No if $\lambda = 0$ is the best choice.
What can you do? Perhaps try to be more refined in what variables to include in the model and better take care of the categorical variables. The above analysis does it naively treating categorical variables as integers which is not that meaningful. In the end, there is no guarantee that Lasso or ridge do better than the linear model.
You can try some non-parametric regression models too. You can also do polynomial regression coupled with lasso to reduce the complexity.
PS. The training MSEs for the above setup are 20.8 for the linear model and the following for Lasso:
Measure: Mean-Squared Error 

    Lambda Index Measure    SE Nonzero
min 0.0174    65   24.78 4.346      13
1se 0.9505    22   28.85 5.214       6

and for the following for ridge:
Measure: Mean-Squared Error 

    Lambda Index Measure    SE Nonzero
min  0.671   100   22.91 2.166      13
1se  3.578    82   25.03 2.229      13

