Loading images directly without a DataLoader should work as long as you are applying the same processing (you wouldn’t need data augmentation).
What kind of transformations did you use during the training, i.e. did you resize the images etc.?
To isolate the issue to the data loading you could load some images during training and compare the predictions to the same images returned by the DataLoader (comparing the raw tensor values could also give you more information about the differences).