I was thinking using the following code in dataloader might help.
RandomCrop will randomly crop the images and the random.randint will give different size for each batch.
I have not written the entire code though.
import random
import torchvision.transforms as tf
img = tf.RandomCrop([random.randint(180,224),random.randint(180,224)])