Conv and pooling layers work on variable spatial input shapes as long as it’s larger than the kernel size.
Conv layers only need the definition of the input channels (defined by the activation input), output channels (number of filters), as well as the kernel size.
Let me know, if I misunderstood your question.