Hi I have a quick question about kl divergence loss in Pytorch


Is it okay to use sigmoid instead of softmax for input? Most of the case I noticed that softmax probability distribution is used, but I wonder does that make sense to use kl divergence loss for multi-label target.


I want to use KL divergence loss by giving more than two dimension input and target. In this case, does KL div loss compare the distritbution between last dimension of input and target? Or does it compare all the dimension except for the batch dim?


thanks