Unfortunately, that is how PyTorch works. You have to specify the input size.
There is a pip package (whose name I am forgetting now). It has this feature that the input dimension is not needed. Plus, it has other things that are missing in PyTorch (like globalaveragepooling2d etc.). I will try to mention the name if I recall (or you can if you find it)