Solved by mailcorahul in post #2 
              
              
                The problem is because you trained on 32x32x3 images. After the 2nd maxpool you get an output of 20x5x5, hence you reshape to x.view(-1, 500). But since MNIST images are 28x28x3 the shape after 2nd maxpool becomes 20x4x4 which after reshaping becomes torch.Size([64, 500]) causing your batch shape ch…