Solved by ptrblck in post #4 
              
              
                The conv and pooling layers in your model will change the shape of your activations depending on the kernel size, stride etc. 
Have a look at CS231n for a detailed description. 
The short version: 


nn.Conv2d with kernel_size=5 and no padding will subtract 2 pixels on each border (4 pixels in width…