Solved by ptrblck in post #3 
              
              
                Usually your data will have the batch dimension in dim0, so using F.log_softmax in dim0, will normalize your log probabilities in this batch dimension, which is most likely wrong. 
Your logits are probably in dim1, so F.log_softmax(tensor, dim=1) should give you the right results.