If you just need to resize your image to a specific or different specific sizes before feeding it to your network, you could do it in a Dataset or  just before the training phase.
AdaptiveAvgPooling is useful, if you have to deal with different sized inputs, which have to be resized to a specific size inside the network (not before!).