Thanks for the reply! I’m actually having this problem even when using the exact same data, no train/test split.
For example, I initialise two copies of the model and load the trained weights into both - in the first I do model1.train(), in the other I do model2.eval(). I then load a single batch of data and pass to both models. model1 will achieve the correct 93% accuracy in this case, as it did in training, as long as the batch size is the same as in training (64). model2 will achieve 50% regardless of batch size.