Multiple things here. The default values for some of the transforms are really bad. I’d look at removing RandomResizedCrop or setting appropriate values for each default parameter, especially scale. Rotations of 15 degrees also seem excessive.
Then, in predict_image, you are removing the color from input (unsure if your data is already grayscale), but more importantly, ToTensor() will divide values by 255, which might not be accounted for when you train without applying the transforms.
Hope that helps.