Hi, as mentioned in the docs, the student logits need to pass through a log_softmax, my question is why? log softmax doesn’t give us a distribution, is it because of the way pytorch implements kl loss?
Thanks!