The problem is probably in the fully-connected layers, as it was expecting some feature map of a specific size and now it’s bigger.
You can bypass that by adding a Adaptive[Max/Average]Pooling2d layer just before the classifier, so that the output size is the same and the classifier can work.