I add a line  in the for loop
print(torch.sum(input_grad_1 - input_grad_2)/len(input_grad_1))

the average grad variance is at 1e-12~-14, I guess it not a big problem? ( for me it is)
and May I ask what is the use to calculate grad on grid?