it depends on what you want to do at the end.
If the convnet is for image classification (you want one output for an image, regardless of size), then you can use nn.AdaptiveAvgPooling right before the fully connected layers.
If you want dense classification (larger image = larger output), you can replace your nn.Linear layers at the end with nn.Conv2d with kernel size 1x1