ptrblck:

k = torch.rand(256)


Thank you @ptrblck
There is something I don’t understand.
Let’s assume that: shape(target) = shape(input) = (batch_size, N)
The log_softmax / softmax must be in dimension 0?
F.kl_div(F.log_softmax(logits, dim = 0), F.softmax(target, dim = 0), reduction="none").mean()
Or in dimension 1 ?
F.kl_div(F.log_softmax(logits, dim = 1), F.softmax(target, dim = 1), reduction="none").mean()
Personally I think in dimension 1 (N here is a bit like the number of classes for a classification)