I assume you are using some pretrained models from torchvision?
If so, the usual input shape is [batch_size, 3, 224, 224] with the exception of Inception, which expects [batch_size, 3, 299, 299].
Since the models use an adaptive pooling layer before flattening the output of the last conv or pooling layer, the spatial size of your input images is more flexible.
However, if you are using a pretrained model, note that the kernels were trained using the aforementioned shapes, so using completely different spatial sizes might result in bad performance.
Let us know, if you have any more questions.