Hey,
I tried to isolate the problem and tried to replicate the issue in a new project, the predictions were consistent as you say.
When I load the pre-trained model, the mode is training by default, which had inconsistent results due to running estimates. I assumed the accuracy of this model is the expected value. When I observed a lower accuracy for the model in eval mode. At the time, I was switching between batch predictions and individual predictions which led to the confusion.
Thank you for checking.
What I still find interesting is that the accuracy drops from 0.703125 (train mode) to 0.515625 (eval mode) for the first batch of validation images in ImageNet. (I hope these numbers are correct)
Bottom line - the eval mode works consistently as expected.
Thanks