Hi Alisson!



 Alef93xx:

Preds: torch.Size([1, 21, 160, 240])
Target: torch.Size([1, 3, 160, 240])
The size of tensor a (21) must match the size of tensor b (3) at non-singleton dimension 1


The short answer is pretty much what the error message says:  You’re
trying to perform an element-wise equality test between two tensors
whose dimensions don’t match.  (If you were comparing two tensors
that had the same shape, you wouldn’t get this error.)
However, I think this is a symptom of some more general structural
things you are are doing wrong …


I’m trying to train a neural network for multiclass semantic segmentation (FCN) using the CrossEntropyLoss function


You don’t show your training code, and I’m a little surprised that you’re
not reporting similar errors with that, given what it looks like you’re
doing wrong.
Anyway, I would expect that you would be using targets (and
inputs) that have the same format for both training and your
accuracy evaluation.
The input to your model, x – the batch of images you wish to
perform semantic segmentation on – would typically have shape
[nBatch, nChannel, height, width].
In your case it appears that nBatch = 1.  (A batch size of 1 is fine.)
It seems that nChannel = 3 – RGB?  But you say that you are using
“RGB 8 bit” which would mean that you would have a single 8-bit
channel with palettized color – that is, you have a look-up table
(LUT) that translates the 8-bit value of each pixel into an RGB color
by using it as an index into the LUT.  So it’s unclear what is going on
with your channel dimension.
height = 160 and width = 240, which is fine.
This line:


y = y.permute(0, 3, 1, 2)



suggests that your images – or at least your target “images” –  are
stored (individually) with shape [height, width, nChannel], so
you permute the dimensions to put them in the order preferred by
pytorch.  This is also fine.  (Why you have the .squeeze(1), I don’t
know.)
The core problem – and this is hard to know for sure because you
don’t show the code or shapes that you use for training – is that
your target (your y) is not formatted correctly for CrossEntropyLoss.
The output of your model (your preds) has shape
[nBatch = 1, nClass = 21, height = 160, width = 240].
This is fine.  But you would then want the target (the annotated
class labels for the image) to have shape [nBatch, height, width],
with no nClass dimension.  Each “pixel” in your target “image”
should be an integer class label that runs from 0 to
nClass - 1 = 20.
It is sometimes the case that target images have the class labels
color coded – that is, all of the class-3 pixels might be colored blue,
all of the class-7 pixels might be colored gray, and so on.  If this is the
case, you would have to pre-process your target images so that
they use integer class labels, as described above.
Now, on to your accuracy calculation:


            preds = nn.Softmax(dim=1)(model(x))



This is okay, but you don’t actually need the Softmax here.
preds = model (x) is simpler, and – for this purpose – equivalent.


            preds = (preds > 0.5).float()



This is incorrect (for the multi-class problem you say you have).
(This would make sense for binary segmentation.)
You want to convert the “soft,” probabilistic prediction made by
your model into a “hard” prediction for the single best class.
For this you want preds = preds.argmax (dim = 1).  This
produces a batch of prediction “images” of shape
[nBatch, height, width] (no nClass dimension) where each
pixel is the integer class label for the class the model predicts
as being the most probable.
If your target (y) is also composed of integer class labels and has
shape [nBatch, height, width], (preds == y).sum() will
count the number of correct predictions.
Best.
K. Frank