I’m not familiar with your use case, but I assume you are working on some regression problem using images.
Based on your model definition, if seems you are increasing the number of channels quite aggressively (going from 1 input channel to 21609 output channels), which is quite uncommon.
Also, nn.MaxPool2d(1) won’t have any effect, as the kernel size is set to 1, which will basically just return the same values. The second ReLU activation also won’t change anything.
I would recommend to increase the number of channels a bit smoother and also to use other base models such as VGG16 as a starter and adapt them to your use case.