Solved by KFrank in post #4 
              
              
                Hello Mr. Robot! 

I’ve haven’t done the experiment, but I very much expect that if 
you train a network on reasonably clean multi-class data (say, 
MNIST digits) with both CrossEntropyLoss and 
BCEWithLogitsLoss you will get better performance using 
CrossEntropyLoss. 
It’s worth saying a few words…