transforms.Resize 3 uses a sequence of sizes or an int:

Desired output size. If size is a sequence like (h, w), output size will be matched to this. If size is an int, smaller edge of the image will be matched to this number. i.e, if height > width, then image will be rescaled to (size * height / width, size)

Since you are providing an int I would assume the smaller edge of the image will be matched. Could this be the issue?