Sklearn most likely is not using first-order gradient descent to solve this. I can’t spot an error in your code, so maybe you just need to add lr decay (scheduler) - in general you should check if your loss decreases at a reasonable pace. Another possible issue is non-normalized data (i.e. epoch 0 prediction is too far off).