@ptrblck I would like to do the same thing with multi-channel inputs and kernels now.
So I have a tensor with shape [N, C, W, H] as the input and a tensor [N, K, kW, kH] where N is the batch size, C is the image channels, and K is the number of kernels and K=C. I want first K filters from the kernel batch to be applied to first image with C channels from the image batch and so on. Each k filter should be only applied one channel c. In the output I should have a feature map with size [N, C, W, H].
Can I do this without a for loop too?