Solved by KFrank in post #2 
              
              
                Hi Arjung! 

The short answer is use BCEWithLogitsLoss. 
Several comments: 
KL divergence and cross entropy are closely related.  For fixed 
targets, KL divergence and cross entropy differ by a constant that 
is independent of your predictions (so it doesn’t affect training). 
You should understand …