I don’t understand why you need using the MSELoss, which is used for regression, while you are doing classification.
You may use instead the CrossEntropyLoss, but don’t forget to add a LogSoftmax layer at the output of your network.
You can find some examples of good practice for semantic segmentation on cityscapes in the Dilated Residual Networks repo by Fisher Yu here 28