Bad results in Logistic Regression from-scratch Python implementation on sample gender data
I am quite a newbie to Machine Learning, now trying to implement from scratch in Python (using Numpy) a logistic regression algorithm.
I took the gender / height / weight data from here: https://github.com/abhiwalia15/500-Person-Gender-Height-Weight-Body-Mass-Index/blob/master/500_Person_Gender_Height_Weight_Index.csv
Then I did the following: 1. Normalized the dimensions using MixMax ([0, 1] range) -the result is in https://drive.google.com/file/d/1yhjyAxbCXAFuh9QJJwcUeMtPonudPyul/view?usp=sharing 2. Replaced Mail / Female by 1/0 in a separate file: the result is in https://drive.google.com/file/d/1h8-7ZTUYUCn333KXYBVVgT7OPBrHZGik/view?usp=sharing
Here is my Python code with some printouts:
``` import numpy as np np.set_printoptions(suppress=True)
def sigmoid(x): return 1 / (1 + np.exp(-x))
def predict(X, W, b): return (sigmoid(X.dot(W) + b))
from numpy import genfromtxt X = genfromtxt('c:\temp\\ml1\data1.csv', delimiter=',',skip_header=1) TT = genfromtxt('c:\temp\ml1\val.csv', delimiter=',',skip_header=1) T = TT.T
b = 2.0 W = np.repeat(1.0, 3)
lr = 0.001 num_epochs = 1000 for epoch in range(num_epochs): Y = predict(X, W, b) W = W - lr * X.T.dot(Y - T) b = b - lr * np.sum(Y - T) print(W) print(b)
print(predict(X, W, b)) ``` I am reading the data (I am aware of losing the 1st line, to overcome some weird issue) - then choosing some learning rate 'lr' and initial values for W and b parameters, then running the algorithm from the "textbook" for 1,000 iterations.
In terms of the convergence, I see that the W and b values are quite stable at the end of the run. Here is the tail of my printouts:
[-0.07538705 -0.06014817 0.19189458] -0.11792230458161261 [-0.07527279 -0.06033511 0.19213628] -0.11806707431160607 [-0.07515862 -0.06052191 0.19237781] -0.11821173599078225 [-0.07504453 -0.06070857 0.19261915] -0.11835628969685595 [-0.07493052 -0.06089509 0.19286032] -0.11850073550750553 [-0.0748166 -0.06108146 0.1931013 ] -0.11864507350037269 [-0.07470277 -0.0612677 0.1933421 ] -0.11878930375306242 [-0.07458902 -0.0614538 0.19358272] -0.11893342634314288 [-0.07447536 -0.06163975 0.19382316] -0.11907744134814532 [-0.07436178 -0.06182557 0.19406342] -0.11922134884556394 [-0.07424828 -0.06201125 0.1943035 ] -0.11936514891285584 [-0.07413487 -0.06219678 0.1945434 ] -0.11950884162744084 [-0.07402154 -0.06238218 0.19478312] -0.11965242706670146 [-0.0739083 -0.06256744 0.19502266] -0.11979590530798276 However, when checking my "predictions" - I see almost all of them close to 0.5, so the algorithm is quite meaningless. 338