When we do 2d convolution with RGB images we are, actually, doing 3d convolution. For this we still use the pytorch 2d_conv layers.
When we do 3d convolution of a set of RGB images, we are doing 4d convolution and can use the 3d conv layer.
My question is: what is the difference, if any, between using the 3d conv layer for a set of grayscale images, as opposed to giving the set of images to pytorch as color channels and using the 2d conv layer?
To be clear, if using 2d conv, my tensor will look like:
[batch size, set of grayscale images, n rows, n cols]
And in the case of 3d conv:
[batch size, set of grayscale images, 1 grayscale channel, n rows, n cols]