Solved by richard in post #4 
              
              
                You’re seeing some floating point error @AlbertZhang 
I ran your code and got the expected values: 
 [image] 
The largest deviation is 2.3842e-07 
In [17]: (x.grad - (z[-1] - y)*5*2/16).max()
Out[17]: tensor(2.3842e-07, grad_fn=<MaxBackward1>)