In the code, is
 loss_manual += -1. * logit[t] + torch.log(torch.sum(torch.exp(logit)))
the equivalent to :
image869×203 15.7 KB
Which works for when the outputs are probabilities summing to 1 (after a softmax layer?)
I will try to reproduce the issue as well.