I’m trying to implement Softmax regression from scratch, but I have a few problems.
The notebook can be visualized at the following link 11, or downloaded directly here 1.
I checked the individual functions and compared the results with the ones PyTorch provides, and they seem correct (i.e., they provide the same values). The problem is that when I train the model, after a few batches the loss becomes NaN. If I replace my implementations of cross-entropy and Softmax, then it seems to be working. So perhaps they are not so correct. It seems that the matrix of weights, W, is NaN already after the first batch.