Body: <p>I'm attempting to train a convolutional neural net to perform binary classification of volumes of shape <span class="math-container">$(H, W, C)$</span> (i.e., height, width, channels). For the sake of this example, let's say that the volumes represent RGB images and that <span class="math-container">$C = 3$</span>.</p>
  <p>My data are formatted such that pixel values are <strong>mutually exclusive across channels.</strong> In other words, if the pixel at <span class="math-container">$H_0, W_0, C_i &gt; 0$</span>, then, all <span class="math-container">$H_0, W_0, C_{j \neq i} = 0$</span>. Said another way, for a given pixel location <span class="math-container">$H_i, W_i$</span>, if the green channel value is greater than 0, the red and blue channel values are always 0.</p>
  <p>The model itself consists of multiple 2D convolutional layers (each followed by max pooling operations), followed by two fully-connected layers.</p>
  <p>Of course, I could always just collapse the channels before training by taking the sum of pixel values across channels (creating an image of shape <span class="math-container">$(H, W, 1)$</span>). However, I would like to keep the channels separate as they represent very different characteristics of the image. Basically, I'd like the network to &quot;learn&quot; the arrangement of non-zero values across the channels. I've also considered doing a <span class="math-container">$1 \times 1$</span> convolution across the multi-channel input as the first step in the model, followed by my various CONV-POOL operations.</p>
  <p>As far as I can tell, this seems like an atypical approach, so I'm wondering if there are obvious issues with training the CNN on mutually-exclusive multi-channel images? Does it even make sense to do this? If available, I'd love to know of any reading material or resources about training CNNs on multi-channel images with mutually exclusive channels. Thanks!</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-09-20T21:02:45.720Z
Comments.Text: so if you have a layer with 10 filters, you want to have 30 channels as output, the first 10 being the R channel convolved with the 10 filters and so on?
Comments.UserDisplayName: Alberto
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-09-21T00:21:48.000Z
Comments.Text: Ah, @Alberto are you referring to something like Depthwise Convolution (as described [here](https://paperswithcode.com/method/depthwise-convolution#:~:text=Depthwise%20Convolution%20is%20a%20type,each%20element%20in%20the%20output.))? I hadn't thought about that, but it might make sense to apply the convolution to each channel separately!
Comments.UserDisplayName: tomsasani
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-09-21T09:59:37.523Z
Comments.Text: I mean, imho, naive convolution might be better, as it is at least as powerful as Depthwise Convolution, however it seemed like it was fitting your requirements
Comments.UserDisplayName: Alberto
Tags: <convolutional-neural-networks><one-hot-encoding><channel>
Title: When training a convolutional neural net, what if the channels in my image are mutually exclusive?
UserDisplayName: tomsasani
ThreadText: # When training a convolutional neural net, what if the channels in my image are mutually exclusive?
  
  (Asked by: username_0 on 2023-09-20)
  
  I'm attempting to train a convolutional neural net to perform binary classification of volumes of shape $(H, W, C)$ (i.e., height, width, channels). For the sake of this example, let's say that the volumes represent RGB images and that $C = 3$.
  
  My data are formatted such that pixel values are **mutually exclusive across channels.** In other words, if the pixel at $H\_0, W\_0, C\_i > 0$, then, all $H\_0, W\_0, C\_{j \neq i} = 0$. Said another way, for a given pixel location $H\_i, W\_i$, if the green channel value is greater than 0, the red and blue channel values are always 0.
  
  The model itself consists of multiple 2D convolutional layers (each followed by max pooling operations), followed by two fully-connected layers.
  
  Of course, I could always just collapse the channels before training by taking the sum of pixel values across channels (creating an image of shape $(H, W, 1)$). However, I would like to keep the channels separate as they represent very different characteristics of the image. Basically, I'd like the network to "learn" the arrangement of non-zero values across the channels. I've also considered doing a $1 \times 1$ convolution across the multi-channel input as the first step in the model, followed by my various CONV-POOL operations.
  
  As far as I can tell, this seems like an atypical approach, so I'm wondering if there are obvious issues with training the CNN on mutually-exclusive multi-channel images? Does it even make sense to do this? If available, I'd love to know of any reading material or resources about training CNNs on multi-channel images with mutually exclusive channels. Thanks!
  
  **Tags:** convolutional-neural-networks, one-hot-encoding, channel
  
  > **Comments:**
  > **username_1**: so if you have a layer with 10 filters, you want to have 30 channels as output, the first 10 being the R channel convolved with the 10 filters and so on?  
  > **username_0**: Ah, @username_1 are you referring to something like Depthwise Convolution (as described [here](https://paperswithcode.com/method/depthwise-convolution#:~:text=Depthwise%20Convolution%20is%20a%20type,each%20element%20in%20the%20output.))? I hadn't thought about that, but it might make sense to apply the convolution to each channel separately!  
  > **username_1**: I mean, imho, naive convolution might be better, as it is at least as powerful as Depthwise Convolution, however it seemed like it was fitting your requirements  
  
  ---
  
  ## 0 answers
  
  