Solved by KFrank in post #3 
              
              
                Hi Charles! 

You are correct that the KL divergence is non-negative, and you are 
also correct that the individual terms returned by pytorch’s “unreduced” 
kl_div() can be negative. 
The reason is that, by definition, the KL divergence is taken between 
two probability distributions, not between tw…