First things first the size is 224 not 244, and I think you should also remove the avgpool layer or just use the nn.AdaptiveAvgPool2d , the input size you choose is recommended to be a  n * 2^5 (n >= 1).