Solved by ptrblck in post #4 
              
              
                Well it’s 4*4*20 = 320. 
You can calculate the shape for the forward pass for each operation. 
While a convolution with kernel_size=5 and no padding shrinks the activation by 4 pixels in height and width, a max pooling of kernel_size=2 and stride=2 pools the activation to half its size. 
From the fo…