I also implemented an UNet variant in pytorch recently, and managed to train it more or less successfully for a Kaggle competition, here it is: https://github.com/lopuhin/kaggle-dstl/blob/292840bf4faf49ecf7c74bed9b6d91982a139090/models.py#L211 834 - but in my case the classes were not mutually exclusive, so I used sigmoid activations.