Yes, these shapes should work.



 R_S:

edit: is there no need for one hot encoding the labels?


No, nn.CrossEntropyLoss expects the target to contain class indices as described in the docs, not one-hot encoded tensors.



 R_S:

can i directly feed the single channel mask (convert color to grayscale) to cross entropy loss function?


No, you should feed the target containing class indices to this loss function. A grayscale mask would have a channel dimension with the size 1, which would not be accepted by nn.CrossEntropyLoss.



 R_S:

now my targets are (16, 256, 256) and the predictions are (16, 12, 256, 256).


These are the right shapes.