The output of the model for a segmentation use case (assuming you are using nn.CrossEntropyLoss) should have the shape [batch_size, nb_classes, height, width], even if not all classes are present in the current crop.
In your case it would thus be [batch_size, 23, height, width], where each channel would represent the logits for the corresponding class.
Based on the error message it seems your channel dimension in the output is smaller than 23.