sami:

What do I assign the output parameters? Any tensor of the size of S?
        torch.autograd.grad(C, S, torch.ones_like(S))



This is how I would understand the docs, since the previous gradients should be multiplied to the ones calculated from your operations.



 sami:

I’ve also tried the following:
        x = copy.copy(self.X)
pself.bias.data *= 0
        pself.weight.data = torch.max(torch.DoubleTensor(1).zero_(), pself.weight)
        Z = pself.forward(self.X) + 1e-9
        S = torch.div(R, Z)

        C = S.backward(torch.ones_like(S))

while printing the .grad of self.X yielded non, the newly copied x variable does have a gradient.


This is weird, since the gradients should be calculated only for the inputs of the operations in the dynamically created graph. Are you sure this is not because of some previous tries which might be still cached or something like that?