Yes sorry I didn’t mean to say logits. I meant softmax probabilities. Yeah, I’ve seen the documentation of the nn.CrossEntropyLoss, I’ll try using that instead of writing my own functon as I was doing up untill now. Also, can you please tell me if the input to the nn.CrossEntropyLoss should be softmax probabilities or raw logits? I’m not very clear after reading the documentation.
And regarding the gradients becoming NaN, I’m just doing torch.zero_grad()…is there anything else…I don’t know.
Thanks for your reply