If you are using nn.CrossEntropyLoss (or nn.NLLLoss), then the target mask shape should be [batch_size, height, width]. I would therefore recommend to transform the binary masks to masks containing the class index (e.g. 0 for background, 1 for mouth and 2 for nose) and add them to a single target mask.