Solved by ptrblck in post #2 
              
              
                Wouldn’t your approach be equivalent to use a nn.Conv2d with a kernel size of (k, 1)? 
This would also only use the height of the kernel and apply it basically to each “column”. 
Let me know, if I misunderstood your use case.