If you didn’t change the training loop besides returning the loss, nothing should change.
How reproducible is this effect? I.e. if you compare the initial code to the modified one (with returned loss) for e.g. 10 runs, what’s the mean and stddev of the final accuracy?