Solved by tom in post #2 
              
              
                z is an intermediate, whose .grad is not usually retained (x,y do have grads). 
If you want the grad for z as well, use z.retain_grad() after z = x + y. 
Best regards 
Thomas