Solved by albanD in post #2 
              
              
                Hi, 
Since you apply the relu inplace in the second case, x now points to the output of the relu. And so you actually do dx/dx = 1. 
If you do the following to have access to the gradient of the original x (before the inplace), it will work. 
x = torch.tensor([-1.0]).requires_grad_()
x2 = x.clone()
…