this might be because of our VGG model (I heard some reports that finetuning gives lower accuracy).
Try https://github.com/jcjohnson/pytorch-vgg 90
It’s converted the Caffe model directly into pytorch format.
These models expect different preprocessing than the other models in the PyTorch model zoo. Images should be in BGR format in the range [0, 255], and the following BGR values should then be subtracted from each pixel: [103.939, 116.779, 123.68]