Solved by tom in post #2 
              
              
                log_softmax gives you logarithms of a distribution… 
Expanding DL(P_target || P_prediction) = P_target * (log P_target - log P_prediction), we see that P_prediction only appears as a logarithm. Given that taking the exponential and then converting back is numerically somewhat unstable, it makes sens…