I’ve downloaded the data and did some checks.
The model output of small stacked batches compared to one large batch (257) is the same.
The losses for batch_size=1 and the unreduced losses for batch_size=257 are the same.
I can’t check the accuracy as I don’t have the state dict, but so far the solution is deterministic.
Also, I had to remove some code, which should be dead anyway due to mode='test', e.g. the transform.