That should be possible. If you stick to certain spatial sizes, e.g. powers of two, I would assume that most UNet implementations can handle the input.
However, if you would like to use arbitrary input shapes, most likely you would have to add some checks and pad/crop manually for odd sizes etc.
The input shape and output shape in the original UNet paper are not matching, if I recall it correctly, but you could implement it in a matching way.
I’ve created a simple UNet style architecture here 442, which could be useful as a code base to implement your model.