Solved by ptrblck in post #4 
              
              
                Yes, Autograd should figure this out. 
Here is a small example, which shows that leaf variables are treated in a special way. I.e. inplace modifications are now allowed, if the value of the variable is needed to calculate the gradient: 
# Both require gradients
x = torch.randn(1, requires_grad=True)…