@ptrblck  Thanks. That certainly makes sense, but did not totally answer my questions though.
Have look at Pytorch official tutorial on transfer learning: https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html 85, which have the following transformations:
data_transforms = {
    'train': transforms.Compose([
        transforms.RandomResizedCrop(224),
        transforms.RandomHorizontalFlip(),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ]),
    'val': transforms.Compose([
        transforms.Resize(256),
        transforms.CenterCrop(224),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ]),
}

Now in the validation transforms, if you replace the two lines transforms.Resize(256),transforms.CenterCrop(224) by simply transforms.Resize((224,224)) then the accuracy drops! Why resizing to 256 and then center-crop 224 better than directly resizing to 224?