How can i avoid over fitting in ridge and lasso regression using the code in the link below?
I used the code provided here: https://towardsdatascience.com/ridge-and-lasso-regression-a-complete-guide-with-python-scikit-learn-e20e34bcbf0b
The only difference is that i used StandardScalar on my data given below:
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform (X_test)
print len(X_test), len(y_test)
Here are my ridge regression results: linear regression train score: 1.0 linear regression test score: -0.07550729376673715 ridge regression train score low alpha: 0.9999999970240117 ridge regression test score low alpha: -0.07532716978805554 ridge regression train score high alpha: 0.8659167364307487 ridge regression test score high alpha: 0.013702748149851396
My Lasso results: training score: 0.48725444995774625 test score: -0.3393210376146986 number of features used: 4 training score for alpha=0.01: 0.9998352085084429 test score for alpha =0.01: -0.6995903332119675 number of features used: for alpha =0.01: 24 training score for alpha=0.0001: 0.9999999830932269 test score for alpha =0.0001: -0.7189894474663594 number of features used: for alpha =0.0001: 25 LR training score: 1.0 LR test score: -0.7217224228737649
I am not able to understand why am i getting such results! Any help is highly appreciated.
Edit: The code is below
#Importing modules
import sys
import math
import itertools
import numpy as np
import pandas as pd
from numpy import genfromtxt
from matplotlib import style
import matplotlib.pyplot as plt
from sklearn import linear_model
from matplotlib import style, figure
from sklearn.linear_model import Lasso
from sklearn.linear_model import Ridge
from sklearn.linear_model import LinearRegression
from sklearn.cross_validation import train_test_split
#Importing data
df = np.genfromtxt('/Users/pfc.csv', delimiter=',')
X = df[0:,1:298]
y = df[0:,0]
print (X).shape
print (y).shape
display (X)
display (y)
print (y)
#print type(newY)# pandas core frame
X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.2,random_state=4)
#Apply StandardScaler for feature scaling
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform (X_test)
print len(X_test), len(y_test)
lr = LinearRegression()
lr.fit(X_train, y_train)
rr = Ridge(alpha=0.01) # higher the alpha value, more restriction on the coefficients; low alpha > more generalization, coefficients are barely restricted and in this case linear and ridge regression resembles
from sklearn.metrics import mean_squared_error
from math import sqrt
rr.fit(X_train, y_train)
rr100 = Ridge(alpha=115.5) # comparison with alpha value
rr100.fit(X_train, y_train)
train_score=lr.score(X_train, y_train)
test_score=lr.score(X_test, y_test)
Ridge_train_score = rr.score(X_train,y_train)
Ridge_test_score = rr.score(X_test, y_test)
Ridge_train_score100 = rr100.score(X_train,y_train)
Ridge_test_score100 = rr100. 392