Hi,
I have a network whose loss depends on both the prediction and the gradients.
In my python 0.3 code I used to do:
grads = torch.autograd.grad(predV, feats, torch.ones_like(predV))
ipdb> grads[0].requires_grad
True
ipdb> predV.requires_grad
True
ipdb> feats[0].requires_grad
True
ipdb> torch.ones_like(predV).requires_grad
False

Notice how the output has requires_grad True?
After some refactoring for 0.4 it suddenly has requires_grad False and now the network will not backprop the error of the gradients anymore.
The weird thing is that I try to replicate it with some toy example in pytorch 0.3 and I cannot.
import torch
x = torch.autograd.Variable(torch.ones((3, 4)), requires_grad=True)
y = x+5
torch.autograd.grad(y, x, torch.ones_like(y))[0].requires_grad
# False

So I would like to know, what exactly makes the output of torch.autograd.grad require gradients?