You have to make sure that the distribution of the test data are similar to the training data.


First, make sure that the sequence of transformations applied to train and test data result in the same range of values. For example, it can be that in the training data, the background is denoted as 1 and character pixels are denoted with non-zero values, but in the test data it’s opposite.
Another thing (which does not seem to be the case based on your code), if you apply transforms.Normalize(mean=0.5, std=0.5), then the input tensors will be in the range [-1, 1] but the other one may be in range [0, 1].


Also, the images in the test set might have completely different background, different scales of the characters with respect to the image size, … So all these can be the reasons why the model does not work well on the test set. So, first you can visualize some samples from train and test sets, and see if there are distinguishable from each other, and try to make them as similar as possible. For example, you can apply thresholding (https://en.wikipedia.org/wiki/Thresholding_(image_processing) 7) to remove the background noise if thats necessary.