as mentioned above VGG16 was trained on much smaller images, so it might not work as good on your data. In order to overcome this issue, you can add conv layer before feeding to VGG16, so the sizes after your conv layer will be at the same scale.