Solved by ptrblck in post #2 
              
              
                As you’ve already pointed out the issue is created by the missing “class dimension” in the output. 
In a multi-class segmentation use case nn.CrossEntropyLoss expects the model output to have the shape [batch_size, nb_classes, height, width], where each “channel” represents an activation maps, which…