Hi gkrisp!



 gkrisp9:

Do you think it’s better to use a resnet with 2d convolutions and override the first layer to have input of 160 channels


It would almost certainly be worse, if I understand correctly that you
are working with 3d images.
The point is that your images have substantive spatial structure.
That is, just as a pixel with x = 17 is right next to x = 18, but far
away from x  = 148, a slice in your 3d “z-stack” with z = 17 is
right next to the z = 18 z-slice, but far away from z = 148.
Convolutions know about and respect this structure, while a general
fully-connected layer does not (and the in_channels → out_channels
part of a convolutional layer is fully connected).
You could use a Conv2d with in_channels = 160 (and with all of
those extra connections, it would be a superset of the Conv3d so,
in principal, could perform the same inference if properly trained),
but such a network would have to “learn” the spatial structure of your
z dimension, which could take a long time or not happen at all with
realistic amounts of training data and time.
As a general rule, if your problem has some particular structure and
there is a natural way of “telling” your network about that structure
by building that structure into the architecture, you’re much better off
doing so, rather than forcing your network to “learn” that structure.


model.features[0] = nn.Conv2d(160,64,7,7,7)



As an aside, if your model were pretrained, so, in particular,
model.features[0] were pretrained, doing this would throw away
that part of the pretraining, which could be costly.





 KFrank:

Your first error was caused because your input didn’t have the nBatch
dimension. This dimension can have size one if you want, but it has to
be present


The input shape is [2,160,256,256] where theoretically 2 is the nBatch.


Yes, my mistake.  You said in your first post that 2 is your batch size.
The basic point still stands, however.  Pytorch’s convolutional layers
require both a batch and a channels dimension, even if they are
“trivial,” singleton (that is, size = 1) dimensions.
So, if your input image has shape [2, 160, 256, 256], with nBatch = 2,
and no explicit channels dimension, you have to add the required
channel dimension, e.g.:
image = image.unsqueeze (1)

However, now you have nChannels = 1, and your pretrained resnet
requires nChannels = 3, so probably the best solution (if you want to
keep the benefits of your pretrained model) is to expand() your input
to have 3 (redundant) channels:
image = image.unsqueeze (1).expand (-1, 3, -1, -1, 1)

Now image will have shape [2, 3, 160, 256, 256].
(Note that expand() just provides a logical view into your 1-channel
image – the 2 additional channels don’t actually “exist” in the sense
of being stored in memory or being able to have values that differ from
those of the one “real” channel.)
Best.
K. Frank