Do I understand correctly from your answer " My evaluation script is pretty standard. with torch.no_grad(): model.eval() …" that you compare the results if the two behave exactly the same at evaluation time?
I am afraid floating point numbers are not associative and doing things in a different order will lead to difference in the result.
While you should not see it for one iteration, when you do training, gradient descent based optimization will increase these small differences and you will most likely converge to a different point.
If you model is well behaved, this other point should have similar properties though so that’s not a problem in ML in general.