Solved by ptrblck in post #6 
              
              
                Yes, that’s right. 

No, if you add another layer (e.g. nn.Conv2d) you would see the channels-last kernels again. The issue with your code would be the reduction, which would drop the memory layout, if I’m not mistaken.