Padding the inputs with shape checks would certainly be another valid approach.
In case you are using transposed conv layers for the upsampling, you could also use the output_size argument (in its forward call) to specify the desired output shape in case it’s ambiguous.