Hi Ereebay!



 Ereebay:

Hi, I have some problems of getting the gradient when I used the kl_div function.
I tried two versions of the kl_div function, one is the official version: F.kl_div, and the other is implemented by myself. The results of the official kl_div and my kl_div are same but the gradient of the F.kl_div is always 0.


I don’t follow fully what you are doing with two layers of losses and grads,
and I will comment in terms of the older 0.3.0 version of pytorch.
I believe what you are seeing is that F.kl_div() doesn’t track gradients
with respect to its second argument (your label).  (I think this is a
“feature” in the interest of efficiency.)  The version of kl_div() that you
wrote isn’t “smart” enough to not compute the gradient with respect
to its second argument.
Please see this related thread:

  
    
    
    Nn criterions don't compute the gradient w.r.t. targets autograd
  
  
    I am implementing a really basic autoencoder in pytorch. 
import torch
import torch.nn as nn
import torch.nn.parallel
import torch.backends.cudnn as cudnn
import torch.optim as optim
from torch.autograd import Variable

BATCH_SIZE = 16

criterion_mse = nn.MSELoss().cuda()
x = Variable(torch.FloatTensor( BATCH_SIZE , 10  )  ).cuda()
l = nn.Linear( 10 , 10 ).cuda() 
y = l(x)
loss = criterion_mse( x , y )

But this code gives the following error. 
AssertionError                            Traceback…
  


Here is a simple pytorch 0.3.0 script that illustrates this for F.kl_div():
import torch
torch.__version__

torch.manual_seed (2020)

# make some log-probabilities and probabilities
logprobs = torch.nn.functional.log_softmax (torch.autograd.Variable (torch.randn (2, 5)), dim = 1).data
probs1 = torch.nn.functional.softmax (torch.autograd.Variable (torch.randn (2, 5)), dim = 1).data
probs2 = probs1.clone()

# make leaf-variable Variables
# input values are log-probabilities
# input  = torch.autograd.Variable (torch.randn (2, 5), is_leaf = True, requires_grad = True), dim = 1)
input = torch.autograd.Variable (logprobs, requires_grad = True)
# target values are probabilities
targ_grad = torch.autograd.Variable (probs1, requires_grad = True)
targ_nograd = torch.autograd.Variable (probs2, requires_grad = False)

loss = torch.nn.functional.kl_div (input, targ_grad)
loss
loss.backward()
input.grad
targ_grad.grad

loss = torch.nn.functional.kl_div (input, targ_nograd)
loss
loss.backward()
input.grad
targ_nograd.grad

And here is the output:
>>> import torch
>>> torch.__version__
'0.3.0b0+591e73e'
>>>
>>> torch.manual_seed (2020)
<torch._C.Generator object at 0x000002904D7C6630>
>>>
>>> # make some log-probabilities and probabilities
... logprobs = torch.nn.functional.log_softmax (torch.autograd.Variable (torch.randn (2, 5)), dim = 1).data
>>> probs1 = torch.nn.functional.softmax (torch.autograd.Variable (torch.randn (2, 5)), dim = 1).data
>>> probs2 = probs1.clone()
>>>
>>> # make leaf-variable Variables
... # input values are log-probabilities
... # input  = torch.autograd.Variable (torch.randn (2, 5), is_leaf = True, requires_grad = True), dim = 1)
... input = torch.autograd.Variable (logprobs, requires_grad = True)
>>> # target values are probabilities
... targ_grad = torch.autograd.Variable (probs1, requires_grad = True)
>>> targ_nograd = torch.autograd.Variable (probs2, requires_grad = False)
>>>
>>> loss = torch.nn.functional.kl_div (input, targ_grad)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: the derivative for 'target' is not implemented
>>> loss
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'loss' is not defined
>>> loss.backward()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'loss' is not defined
>>> input.grad
>>> targ_grad.grad
>>>
>>> loss = torch.nn.functional.kl_div (input, targ_nograd)
>>> loss
Variable containing:
 0.2735
[torch.FloatTensor of size 1]

>>> loss.backward()
>>> input.grad
Variable containing:
1.00000e-02 *
 -0.6291 -1.5081 -5.4165 -1.8988 -0.5475
 -0.8674 -0.3891 -0.4075 -0.6592 -7.6768
[torch.FloatTensor of size 2x5]

>>> targ_nograd.grad
>>>

The pytorch 0.3.0 version of F.kl_div() simply refuses to accept a second
argument that has requires_grad = True, and throws an error.  If the
second argument has requires_grad = False, F.kl_div() works, and
you get a gradient for its first argument, but, of course, not its second.
Does this look like the root cause of what you are seeing?
Best.
K. Frank