Solved by tom in post #2 
              
              
                At the risk of being the CTC person that I don’t want to be: 
The key that the paper is not using log space, so we are not interested in the derivative by y, but by log y. With this, we can remind ourselves that is that log probs are logits (i.e. x.log_softmax(dim=1) == x.log_softmax(dim=1).log_soft…