Convolutions are spatial operations and they do not require fixed size. Ideally, you should be able to use any size since if the dataset is big enough, network will be used to any object size inside the image.  As convolutions are spatial operations you are generating local features, thus, it doesn’t really matter the image size at all.
BTW last layer is nothing but  a mapping (still fully connected network?) which is the most restrictive part.