@Ismail_Elezi
I am using this code to test the behaviour of KLDivLoss. I am using same tensor data for my input and target. So, I am expecting the loss to be zero.
    rand_data = torch.randn(1,1000)

    a = Variable(rand_data)
    b = Variable(rand_data)

    a_lsm = F.log_softmax(a)
    b_sm = F.softmax(b)
    
    criterion = nn.KLDivLoss()

    loss = criterion(a_lsm,b_sm)

    print(loss)

But when I run it a few times, it gives me very small numbers as outputs (both positive and negative). Can someone tell me if I am making a mistake here?
These are some of my outputs:
Variable containing:
1.00000e-12 *
1.4934
[torch.FloatTensor of size 1]
Variable containing:
1.00000e-11 *
1.8763
[torch.FloatTensor of size 1]
Variable containing:
1.00000e-11 *
-2.3461
[torch.FloatTensor of size 1]