Hi Szandala!



 szandala:

Tho I am surprised I cannot use Softmax.


As per the documentation, CrossEntropyLoss has Softmax built in.
This is for reasons of numerical stability and convenience.  If you had
a Softmax at the end of your network, you would be running your
predictions through Softmax twice, which is not what you want.


SO I should remove now Softmax and append it when I will be using the network?


You should remove the final Softmax from your model.
But, in general, you do not need Softmax for anything, so you normally
won’t want to append it when you are “using the network.”
Your network will emit raw-score logits as its predictions.  If you pass
them through Softmax you will get probabilities.  But the logits and
probabilities contain the same information, so, in general, you can work
directly with the logits and not need to convert them to probabilities.
Note that the argmax() of a set of logits will be the same as that of
the corresponding set of probabilities.  So if you make a firm “class-X”
prediction by taking the argmax() of your probabilities, your can
equivalently take the argmax() of the logits.
Best.
K. Frank