I am not sure about the use case.
But do you think, the losses from the images with “less mask area” should be weighed higher? If not, the model might get away by ignoring them.
i.e.,
output = model(input)
abs_diff = torch.abs((input - output)*mask).sum([1,2,3])
loss = abs_diff / (mask.sum([1,2,3]) + eps)  # eps ~ 10-6