Hi Hari!



 Hari_Krishnan:

I’m trying to create my own network to train on the iris dataset. But the loss is not decreasing after 10000 iterations
...
sig = nn.Softmax(dim = 0)
model = torch.nn.Sequential(out1,relu1,out2,relu2,out3,sig)
loss_fn = nn.CrossEntropyLoss()
...



One error is that you are using a Softmax after your last Linear
layer, even though you are using CrossEntropyLoss for your loss.
CrossEntropyLoss has (in effect) Softmax built into it.  That is, it
expects unnormalized, “raw” scores (logits) for its predictions, rather
than probabilities (as produced by Softmax).
Try your model without the final Softmax layer:
model = torch.nn.Sequential(out1,relu1,out2,relu2,out3)

(I haven’t looked at the rest of your code in any detail; there may be
other issues.)
[Edit:  And another thing …   It doesn’t matter, because you shouldn’t
be using Softmax, but if you were using Softmax, it should be
Softmax (dim = 1).  dim = 0 is your batch dimension, while dim = 1
is your class (category) dimension.  For each sample in your batch
you want your per-class probabilities summed over classes to be equal
to 1.  That’s what Softmax (dim = 1) will give you.]
Good luck!
K. Frank