Solved by ptrblck in post #8 
              
              
                No, you don’t need to add the adaptive pooling layer at the beginning of the model. 
vgg16 uses this layer internally before feeding the activation to the linear layer in this line of code. 
You can just use different shapes: 
model = models.vgg16()

x = torch.randn(1, 3, 224, 224)
out = model(x)

…