Haven’t checked the gradient clipping part but the negative loss occurs because you need to use LogSoftmax with the NLLLoss. If the output vector is y^ = [0.99 0.01 0.0]  (which is the output of softmax) and the true class y = 0 the NLLLoss is simply defined as -0.99. If you use log softmax for the same output vector defined above then y^ = [0 -4.6 -inf] so NLLLoss will be -0 = 0.