I don’t think this approach would work out of the box with the current implementations without splitting the operations or alternatively zeropadding the filter kernels in the unwanted input channel dimensions.