@ptrblck Coming back to this.
I am able to pass much larger images and as you said, the adaptive pool layer before the fully connected layer allows for this.
What do you mean by fine-tuning the model? In terms of fine-tuning in the transfer learning domain or something more structural. For example, I was thinking adding a convolutional layer at the beginning of the pretrained network, followed by an AdaptiveAvgPool2d layer to bring the output size to 224x224 and then pass it into the next conv layer, which is in fact the first of the pretrained layer. Does this make sense?
Thanks!