Hi, thank you for the response and the code. If you change the third to last and second to last line to:
print (input.grad/4)
grad2 = input.grad/4

You will get the equal gradients. So this suggests its a scaling issue and tuning the learning rate should solve it rather than a more fundamental issue where the mean of the minibatch losses doesn’t equal the total batch loss.