In this case you probably want RandomResizedCrop instead: torchvision.transforms — Torchvision 0.11.0 documentation 3
Assuming your source images are 28x28, 5 pixels or 23x23 means a minimum scale of ~0.675. Could you check the output with transforms.RandomResizedCrop(input_size, scale=(0.675, 1.0)) is what you’re looking for?