Since you are passing the targets as LongTensors, I assume you are using nn.CrossentropyLoss.
Also based on the posted architecture it seems you are working on a multi-class segmentation use case.
If that’s the case, the model output is expected to contain logits (so remove the sigmoid) in the shape [batch_size, nb_classes, height, width], while the target should be a LongTensor in the shape [batch_size, height, width] containing the class indices in the range [0, nb_classes-1].