Inputs in the same shape make it often easier to feed them as batches to the model.
Otherwise you would have to resize or pad them in order to create batches.
If you have the ability to resize them now to a static shape, I would try it in case you don’t have strong arguments against it.