I come across some issues with getting the gradient with respect to the input in pytorch 0.4. In older version I would do something as in this post

  
    
    
    Strange behavior of Variable.cuda() and Variable.grad 
  
  
    In the first example a is the result of the cuda() method call on a user-created Variable. i.e. Variable(torch.randn(2,10), requires_grad=True) is a leaf, but Variable(torch.randn(2,10), requires_grad=True).cuda() is a different variable, and is not.
  


but in pytorch0.4, since variable is deprecated, the following code will not work.
In [1]: import torch

In [2]: inp = torch.rand(3, requires_grad=True).to(0)

In [3]: l = torch.nn.Linear(3,4).to(0)

In [4]: o = l(inp)

In [5]: o.backward(torch.ones_like(o).to(0))

In [6]: list(l.parameters())[0].grad
Out[6]: 
tensor([[ 0.0948,  0.7947,  0.0521],
        [ 0.0948,  0.7947,  0.0521],
        [ 0.0948,  0.7947,  0.0521],
        [ 0.0948,  0.7947,  0.0521]], device='cuda:0')

In [7]: inp.grad

In [8]: 

However, the following code will work and I am confused, has anyone got an answer to this behavior in pytorch0.4?
In [1]: import torch

In [2]: inp = torch.rand(3, requires_grad=True)

In [3]: l = torch.nn.Linear(3,4).to(0)

In [4]: o = l(inp.to(0))

In [5]: o.backward(torch.ones_like(o).to(0))

In [6]: inp.grad
Out[6]: tensor([-0.7010, -0.2453, -1.1455])