I believe every channel of let’s say an RGB image contribute to the overall properties of that image meaning all of the channels must be taken into account when performing the convolution process. Let’s say you just have one filter, a convolution process applies that filter into all of the 3 channels which will output 3 difference results of same dimension and those 3 results are added.