Even-valued kernel sizes, such as 4, are generally avoided. This is because they have symmetry. Odd-valued kernels are preferred because the lack of symmetry causes there to be a central value, and so tends to be better at providing visual clarity.


You’re using batchnorm2d but not setting the Conv2d layer biases to False. At best, this is adding extra unnecessary calculations because, for example:


4x+3+6 = 4x+9
And at worse, you might be causing your first few layers to have vanishing gradients, due to how many sequential layers gradient descent must work through. Batchnorm2d contains a bias, and so the 2 biases back to back are redundant.

It’s one big sequential, which means that the gradients might be getting infinitesimally small by the time they back propagate to the first layer. You can address this by either reducing the overall layers, thru including some MaxPool2d layers (2,2 stride, kernel is a popular choice). Or you can add split connections.

Your dropout is also quite low, have you tried playing with p=0.4 or 0.5?