Torcione:

May I ask you how did you guess the order of magnitude of the difference in the 2 cases? (Sorry for the stupid question)


That’s the typical relative error in float32, so it was just my guess as you would see it in a lot of other operations as well.



 Torcione:

Is it possible to pass to a double precision for the gradients component in order to verify that this is the right source of error? (in that case the order of magnitude of the difference should decrease and, in particular, it should be possible to predict the new order of magnitude as you just did for the floating precision)


Yes, absolutely! If you are using float64 you should see a relative error of ~1e-13 or so (my gut feeling is way worse for float64  ).