The output of your fc2 layer is defined as 32, so I’m not sure why you assume it should be 40?
Anyway, your model architecture won’t currently work, since the output activation of conv3 will have the shape [batch_size, 32, 1, 1], so the number of input features to fc1 should be 32 instead of 512.