Answers.Body: <p>You can do whatever you want, but it won't necessarily achieve a good result. CNNs have the inductive bias that features are spatially correlated. If you simply stack different views and immediately pass it through a convolutional layer, your model likely won't get a chance to actually extract information from a particular view (via iteratively building up every individual feature's receptive field) before it is combined with a different view (vanilla CNNs are not good approximating identity functions). Additionally, you should not be training your models from scratch. What would likely be more effective is to use a pre-trained model on your different views to extract feature maps corresponding to each view. Then, use those features in a fully connected layer to predict scores you can then input into your softmax.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-10T18:52:26.510Z
Answers.Text: I know use pre-trained model, but what do you mean extract feature maps corresponding to each view. Does it mean I shall train with input (600,800,1) to the output (600,800,5)?
Answers.UserDisplayName: Muhammad Ikhwan Perwira
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-12T01:06:22.250Z
Answers.Text: @MuhammadIkhwanPerwira Not exactly. Many of the pretrained backbones expect the channel dimension to be of size 3, so you should duplicate one view in the channel dimension 3 times.
Answers.UserDisplayName: Jayson Meribe
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-12T06:56:31.623Z
Answers.Text: I see, so what you mean by backbones is basically input layer that has dimension (600, 800, 3).
Answers.UserDisplayName: Muhammad Ikhwan Perwira
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-12T06:57:55.100Z
Answers.Text: So, you think I shall train the model with mixed view? Instead of `(num of sample, height, width, num of views)`, I must train with this input shape `(num of sample X num of views, height, width, 3)`.
Answers.UserDisplayName: Muhammad Ikhwan Perwira
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-10T12:45:34.840Z
Answers.LastActivityDate: 2024-08-10T12:45:34.840Z
Answers.UserDisplayName: Jayson Meribe
Body: <p>I have grayscaled image, specifically medical data ultrasonography.</p>
  <p>In the context of medical domain, there are techniques to capture data that called &quot;View&quot;. It's like point of views of CCTVs with different placement when capturing image. So, there are 5 views actually. Therefore, there are 5 image captured at once.</p>
  <p>Since the image is grayscale which indicate one channel, can I replace the num of channels to the num of views.</p>
  <p>So the input dimension will looks like this:</p>
  <ul>
  <li>(Height, Width, Channel/View) -&gt; (600, 800, 5)</li>
  </ul>
  <p>Instead of this:</p>
  <ul>
  <li>(Height, Width, Channel/Color) -&gt; (600, 800, 1)</li>
  </ul>
  <p>Or, shall I use Conv3D for this? even though my data doesn't contains Z-axis.</p>
  <ul>
  <li>(Depth/View, Height, Width, Channel/Color) -&gt; (5, 600, 800, 1)</li>
  </ul>
  <p>At the end, the output dimension is a vector with 3 element. (three neurons) with softmax activation as the classifier of pathology/disease/diagnose.</p>
  
Tags: <deep-learning><keras><architecture><input-layer><dimensionality>
Title: What is the channel dimension other than color representation in Conv2D? Shall I use Conv3D instead?
UserDisplayName: Muhammad Ikhwan Perwira
ThreadText: # What is the channel dimension other than color representation in Conv2D? Shall I use Conv3D instead?
  
  (Asked by: username_0 on 2024-08-09)
  
  I have grayscaled image, specifically medical data ultrasonography.
  
  In the context of medical domain, there are techniques to capture data that called "View". It's like point of views of CCTVs with different placement when capturing image. So, there are 5 views actually. Therefore, there are 5 image captured at once.
  
  Since the image is grayscale which indicate one channel, can I replace the num of channels to the num of views.
  
  So the input dimension will looks like this:
  
  * (Height, Width, Channel/View) -> (600, 800, 5)
  
  Instead of this:
  
  * (Height, Width, Channel/Color) -> (600, 800, 1)
  
  Or, shall I use Conv3D for this? even though my data doesn't contains Z-axis.
  
  * (Depth/View, Height, Width, Channel/Color) -> (5, 600, 800, 1)
  
  At the end, the output dimension is a vector with 3 element. (three neurons) with softmax activation as the classifier of pathology/disease/diagnose.
  
  **Tags:** deep-learning, keras, architecture, input-layer, dimensionality
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-08-10. Score: 1 | Accepted answer
  
  You can do whatever you want, but it won't necessarily achieve a good result. CNNs have the inductive bias that features are spatially correlated. If you simply stack different views and immediately pass it through a convolutional layer, your model likely won't get a chance to actually extract information from a particular view (via iteratively building up every individual feature's receptive field) before it is combined with a different view (vanilla CNNs are not good approximating identity functions). Additionally, you should not be training your models from scratch. What would likely be more effective is to use a pre-trained model on your different views to extract feature maps corresponding to each view. Then, use those features in a fully connected layer to predict scores you can then input into your softmax.
  
  > **Comments:**
  > **username_0**: I know use pre-trained model, but what do you mean extract feature maps corresponding to each view. Does it mean I shall train with input (600,800,1) to the output (600,800,5)?  
  > **username_1**: @MuhammadIkhwanPerwira Not exactly. Many of the pretrained backbones expect the channel dimension to be of size 3, so you should duplicate one view in the channel dimension 3 times.  
  > **username_0**: I see, so what you mean by backbones is basically input layer that has dimension (600, 800, 3).  
  > **username_0**: So, you think I shall train the model with mixed view? Instead of `(num of sample, height, width, num of views)`, I must train with this input shape `(num of sample X num of views, height, width, 3)`.  
  
  ---
  
  