It seems like AdaptiveAvgPool2d will return whatever height and width as you pass as input, which in your case is 1. It is not clever enough to infer that you want to discard the empty dimensions as well…
So if that is your wish, you should write a custom module which loads the resnet50 layers, removes the undesired ones, and adds yours to it, then in the forward() method of your custom module, call x = torch.squeeze(x) after the AdaptiveAvgPool2d layer (which you would have defined in the __init__() method).
Using the Sequential API makes it harder to debug such cases, so I’d recommend switching to inheriting the nn.Module API with a custom module and debugging from there, so that you could print the shape of the tensor before the AdaptiveAvgPool2d layer!
After trying an AdaptiveAvgPool2d layer on a [3, 16, 7, 7]-shaped tensor on my machine, I can confirm that it works correctly and outputs a [3, 16, 1, 1]-shaped tensor.