Most likely you’ll get the tensor shapes for the tensors where the mismatch occurred.
In your case, it’s:

RuntimeError: size mismatch, m1: [1 x 26624], m2: [1664 x 1] at …

If you change the in_features of your linear layer to 26624, your code will run successfully with 320x320 shaped inputs.
Another approach would be to print the shape of the incoming tensors using this small module:
class Print(nn.Module):
    def __init__(self):
        super(Print, self).__init__()
        
    def forward(self, x):
        print(x.shape)
        return x

model.classifier = nn.Sequential(
    Print(),
    nn.Linear(in_features=26624, out_features=1),
    nn.Sigmoid()
)