Solved by ptrblck in post #2 
              
              
                CrossEntropyLoss does not work with one-hot encoded targets. 
The target dimensions are specified as [batch_size]. Internally the indices are stored. 
In the example input has a batch size of 3 and provides 5 logits. 
The target has the same batch size of 3 with random indices.