I’m currently trying to modify the VGG16 network architecture so that it’s able to accept 400x400 px images.
Based on literature that I’ve read, the way to do it would be to covert the fully connected (FC) layers into convolutional (CONV) layers. This would essentially " allow the network to efficiently “slide” across a larger input image and make multiple evaluations of different parts of the image, incorporating all available contextual information." Afterwards, an Average Pooling layer is used to “average the multiple feature vectors into a single feature vector that summarizes the input image”.
I’ve done this, and have come up with the following network architecture:
----------------------------------------------------------------
        Layer (type)               Output Shape         Param #
================================================================
            Conv2d-1         [-1, 64, 400, 400]           1,792
              ReLU-2         [-1, 64, 400, 400]               0
            Conv2d-3         [-1, 64, 400, 400]          36,928
              ReLU-4         [-1, 64, 400, 400]               0
         MaxPool2d-5         [-1, 64, 200, 200]               0
            Conv2d-6        [-1, 128, 200, 200]          73,856
              ReLU-7        [-1, 128, 200, 200]               0
            Conv2d-8        [-1, 128, 200, 200]         147,584
              ReLU-9        [-1, 128, 200, 200]               0
        MaxPool2d-10        [-1, 128, 100, 100]               0
           Conv2d-11        [-1, 256, 100, 100]         295,168
             ReLU-12        [-1, 256, 100, 100]               0
           Conv2d-13        [-1, 256, 100, 100]         590,080
             ReLU-14        [-1, 256, 100, 100]               0
           Conv2d-15        [-1, 256, 100, 100]         590,080
             ReLU-16        [-1, 256, 100, 100]               0
        MaxPool2d-17          [-1, 256, 50, 50]               0
           Conv2d-18          [-1, 512, 50, 50]       1,180,160
             ReLU-19          [-1, 512, 50, 50]               0
           Conv2d-20          [-1, 512, 50, 50]       2,359,808
             ReLU-21          [-1, 512, 50, 50]               0
           Conv2d-22          [-1, 512, 50, 50]       2,359,808
             ReLU-23          [-1, 512, 50, 50]               0
        MaxPool2d-24          [-1, 512, 25, 25]               0
           Conv2d-25          [-1, 512, 25, 25]       2,359,808
             ReLU-26          [-1, 512, 25, 25]               0
           Conv2d-27          [-1, 512, 25, 25]       2,359,808
             ReLU-28          [-1, 512, 25, 25]               0
           Conv2d-29          [-1, 512, 25, 25]       2,359,808
             ReLU-30          [-1, 512, 25, 25]               0
        MaxPool2d-31          [-1, 512, 12, 12]               0
           Conv2d-32           [-1, 4096, 1, 1]     301,993,984
             ReLU-33           [-1, 4096, 1, 1]               0
          Dropout-34           [-1, 4096, 1, 1]               0
           Conv2d-35           [-1, 4096, 1, 1]      16,781,312
             ReLU-36           [-1, 4096, 1, 1]               0
          Dropout-37           [-1, 4096, 1, 1]               0
           Conv2d-38              [-1, 3, 1, 1]          12,291
AdaptiveAvgPool2d-39              [-1, 3, 1, 1]               0
          Softmax-40              [-1, 3, 1, 1]               0
================================================================
Total params: 333,502,275
Trainable params: 318,787,587
Non-trainable params: 14,714,688
----------------------------------------------------------------
Input size (MB): 1.83
Forward/backward pass size (MB): 696.55
Params size (MB): 1272.21
Estimated Total Size (MB): 1970.59
----------------------------------------------------------------

My question is simple: Is the use of the average pooling layer at the end necessary? It seems like by the last convolutional layer, we get a 1x1 image with 3 channels. Doing an average pooling on that would seem to not have any effect.
If there is anything amiss in my logic/ architecture, kindly feel free to point it out.
Thanks!