newbieatpytorch:

So default conv2d layer will reduce the size on input by 2 for both dimensions and maxpooling will floor-half the input


Yes, that is correct, if your Conv2d has a stride of one a 0 padding.
What you need to do to resolve your problem is x = torch.squeeze(x) just before your call to self.linear1(x). This is because x, before the squeeze, has a shape of B x 32 x 1 x 1 and by squeezing it, the shape will become B x 32 which will be compatible with your Linear layer (B being the batch size).
Also, (this does not change anything), but you use self.pool3(x) 2 times during your forward pass