Solved by ptrblck in post #2 
              
              
                nn.CrossEntropyLoss expects the target as a LongTensor containing the class indices rather than a one-hot encoded target. 
Since you are indexing the logits for the target of the corresponding sample, you can use the class indices directly. 
Have a look at the docs for more information.