Hi Ömer!



 jarvico:

How come pure python implementation resulted in a much better performance than my first Pytorch model? I really don’t know.


I don’t know either.
But somewhere your pytorch code and numpy code are doing
something different.
I would suggest looking at a single forward and backward pass,
and see where things first differ.  Make sure you feed the same
sklearn.datasets into both your pytorch and numpy networks.
Make sure both networks are initialized with the same weights
and biases.
Run a batch of inputs through both networks.  Do you get the same
predictions?  If not, compare intermediate results until you localize
the difference.
If the predictions are the same, do you get the same loss value?
Run your backward pass.  Do you get the same gradients?  If not
look at intermediate results, etc.
There is no reason you shouldn’t be able to get the same results
with pytorch and numpy (up to reasonable round-off error, which
isn’t your issue here).
(As an aside, I don’t see your numpy code backpropagating through
the softmax calculation in your model’s forward() function.)
Good luck.
K. Frank