So the errors form the nondeterminism in the PyTorch interpolate function are very small (they’re mostly from numerical precision of different orders of adding floating point numbers). Adam has the somewhat unfortunate property that the first few steps have funny scaling which can amplify random fluctuations. So you might add an Adam warmup phase with a very low learning rate.
Note that in general I would consider your results stable enough to embrace the fact that there is stochastic fluctuation. The rather weak reproducibility that PyTorch can give you is no replacement for stable procedures as other people with different hardware, different OS, different tooling would want to be able to reproduce your results, too.
Best regards
Thomas